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Abstract 

The  problem  being  addressed  is  how  to  best  find  and  engage  an  unknown  number  of 
targets  in  unknown  locations  (some  moving)  using  multiple  autonomous  wide  area  search 
munitions.  In  this  research  cooperative  behavior  is  being  investigated  to  improve  the 
overall  mission  effectiveness.  A  computer  simulation  was  used  to  emulate  the  behavior  of 
autonomous  wide  area  search  munitions  and  measure  their  overall  expected  performance. 
This  code  was  modified  to  incorporate  the  capability  for  cooperative  engagement  based  on 
a  parameterized  decision  rule.  Using  Design  of  Experiments  (DOE)  and  Response  Sinrface 
Methodologies  (RSM),  the  simulation  was  run  to  achieve  optimal  decision  rule  parameters 
for  given  scenarios  and  to  determine  the  sensitivities  of  those  parameters  to  the  precision 
of  the  Autonomous  Target  Recognition  (ATR)  algorithm,  guidance  precision  and  lethality 
of  the  warhead,  and  the  characteristics  of  the  battlefield. 

Results  show  that  the  form  of  cooperative  engagement  used  in  this  study  is  most 
useful  in  overcoming  the  limitations  on  warhead  lethality  and,  to  a  lesser  degree,  probability 
of  target  report  (Ptr)-  However,  cooperative  engagement  alone  is  not  able  to  compensate 
for  higher  false  target  attack  rates.  Also,  the  selection  of  the  optimal  weights  in  the  decision 
algorithm  are  very  sensitive  to  all  battlefield  characteristics. 
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COOPERATIVE  BEHAVIOR  SCHEMES  FOR  IMPROVING  THE 
EFFECTIVENESS  OF  AUTONOMOUS  WIDE  AREA  SEARCH 

MUNITIONS 


I.  Introduction 


1.1  General 

The  problem  being  addressed  is  how  to  best  find  and  engage  an  unknown  number 
of  targets  in  unknown  locations  (some  moving)  using  multiple  cooperating  autonomous 
wide  area  search  munitions.  The  problem  is  exacerbated  by  the  fact  that  not  all  target 
priorities  are  the  same,  the  munition  target  discrimination  capability  is  never  perfect,  and 
target  destruction  is  never  a  certainty  even  once  engaged-  Further,  factors  such  as  clutter 
density  throughout  the  battlefield  and  ratio  of  targets  to  civilian  or  military  non-targets 
create  even  more  complications  for  these  smart,  yet  simple-minded,  munitions. 

This  research  does  not  provide  a  precise  solution  to  this  rather  ambiguous  and  com¬ 
plex  problem;  rather,  this  research  provides  a  possible  methodology  for  how  to  attack  this 
problem  using  different  optimization  methodologies  and  shows  some  sample  results. 

This  research  was  sponsored  by  the  Munitions  Directorate  of  the  Air  Force  Research 
Laboratory  at  Eglin  (AFB).  All  research  took  place  at  the  Air  Force  Institute  of  Technology 
(AFIT),  Wright-Patterson  AFB  ,  Ohio. 

1.2  Background 

The  United  States  Air  Force  has  significantly  reduced  the  size  of  its  military  forces  as 
a  response  to  changing  national  military  objectives  and  diminishing  budgets.  This  reality 
has  forced  the  Air  Force  to  look  for  more  cost  effective  ways  of  achieving  its  extremely 
crucial  mission.  One  advancement  has  been  the  development  of  small,  lightweight,  low-cost, 
autonomous  munitions  fully  equipped  with  INS /GPS  navigation  and  seekers  capable  of 
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Autonomous  Target  Recognition  (ATR).  The  intent  in  using  these  autonomous  munitions 
is  to  employ  larger  numbers  of  cheaper,  less  sophisticated  munitions  as  opposed  to  fewer 
numbers  of  expensive,  complex  mxmitions.  However,  in  order  to  realize  the  full  capabilities 
of  a  system  composed  of  large  numbers  of  smaller  subsystems  (or  agents),  the  individual 
agents  must  behave  cooperatively  Methods  of  evaluating  mission  effectiveness  of  these 
munitions  have  previously  been  developed  for  the  case  of  non-cooperating  munitions.  In 
this  research  cooperative  behavior  is  being  investigated  to  improve  the  overall  mission 
effectiveness. 

Both  the  Air  Force  and  Army  are  very  interested  in  evaluating  the  expected  effec¬ 
tiveness  of  the  inclusion  of  cooperative  behavior  in  these  systems.  The  Air  Force’s  primary 
wide  area  search  munition  in  laboratory  development  is  the  Low  Cost  Autonomous  At¬ 
tack  System  (LOCAAS).  The  Army,  in  collaboration  with  the  Defense  Advance  Research 
Projects  Agency  (DARPA),  is  investigating  similar  technologies  with  their  Net  Fires  pro¬ 
gram.  Systems  such  as  this  are  specifically  designed  to  autonomously  detect,  identify  and 
destroy  a  number  of  different  types  of  mobile  and  relocatable  targets.  The  primary  targets 
to  be  destroyed  are  Surface-to-Air  Missile  (SAM)  sites,  tactical  Surface-to-Sxu:face  Missile 
(SSM)  launchers,  and  interdiction  targets  such  as  tanks  and  artillery.  The  main  difference 
between  the  Air  Force  and  Army  systems  is  that  they  are  deployed  from  aircraft  and  groxmd 
launchers,  respectively.  Conceptually,  both  systems  would  deploy  multiple  munitions  or 
submunitions  which  would  execute  complementary  search  patterns  once  they  reached  the 
target  area.  Current  technology  is  limited  to  autonomous  search  and  engagement  on  the 
part  of  the  submunitions,  but  it  has  long  been  suspected  that  overall  system  effectiveness 
could  be  improved  through  cooperative  behavior. 

In  a  study  performed  by  RAND  [7],  a  rationale  was  developed  for  investigating  co¬ 
operative  behavior  between  Proliferated  Autonomous  Weapons  (PRAWNs).  They  showed 
by  implementing  a  cooperative  weapon  behavior  logic  into  a  computer  simulation  that 
there  was  a  definite  added  potential  when  cooperation  was  incorporated  into  the  logic  of 
PRAWNs.  This  study  supported  the  hypothesis  that  while  the  individual  weapons  may 
be  less  capable  than  conventional  weapons  under  development  today,  through  commimicar 
tions  across  the  swarm  of  weapons,  the  group  exhibits  behaviors  and  capabilities  that  can 
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exceed  those  of  more  conventional  systems  that  do  not  employ  commrmications  between 
weapons.  The  benefits  which  come  about  through  shared  knowledge  include  possible  re¬ 
laxed  sensor  performance  requirements,  robustness  to  increcises  in  Target  Location  Errors 
(TLE),  and  adaptivity  to  attrition  and  poor  target  characterization. 

In  this  study,  however,  a  fixed  decision  rule  (called  “swarming  algorithm”)  was  used. 
This  eilgorithm  was  based  on  the  foundations  of  two  areas  of  study:  ethology  (the  science 
of  animal  behavior)  and  robotics.  The  collective  intelligence  that  seems  to  emerge  from 
what  are  often  large  groups  of  relatively  simple  agents  is  what  the  engineers  of  the  RAND 
study  tried  to  capture  in  their  swarming  algorithm.  While  this  algorithm  worked  for 
what  they  were  doing,  the  research  did  not  show  how  this  decision  algorithm  compared  to 
other  possible  decision  algorithms.  Also,  the  RAND  study  concentrated  on  a  very  specific 
battlefield  layout  that  was  composed  of  large  clusters  of  targets  and  no  possibility  of 
encounters  with  non-targets  or  clutter.  By  not  taking  into  account  non-targets  or  clutter, 
the  mimitions  had  no  false  target  attacks.  According  to  Jacques  [9],  methods  and  models 
for  evaluating  the  effectiveness  of  wide  area  search  mimitions  must  take  into  account  the 
degradation  due  to  false  target  attacks. 

1.2.1  Ethology.  Scientists  studying  animal  behavior  have  identified  and 
analytically  modeled  many  behaviors  of  natural  organisms  that  have  parallels  to  the  tasks 
that  weapons  must  achieve  in  order  to  search  for,  acquire,  and  attack  targets.  These  tasks 
include  cooperative  search,  cooperative  engagement,  protection  of  each  individual  agent, 
2ind  optimal  path  planning.  Pairticularly,  those  animals  which  exhibit  improvement  in  task 
performance  when  they  interact  as  a  group  are  of  the  highest  interest.  These  include  (but 
cure  certainly  not  limited  to)  bird  flocks,  animal  herds,  schools  of  fish,  bee  swarms,  and  ant 
colonies.  When  looked  at  as  aggregate  motions,  these  behaviors  are  usually  referred  to  as 
swarm  behaviors. 

Reynolds  considered  the  formation  of  flocks,  herds,  and  schools  in  simulations  in 
which  multiple  autonomoiis  agents  were  repulsed  by  one  another  and  attracted  to  one 
another  (and  other  foreign  objects)  by  inverse  square  law  forces  [16].  He  believed  each 
agent  (bird,  einimal,  or  fish)  was  responding  only  to  its  limited-range  local  perception  of 


1-3 


the  world  and  that  natural  flocks  seemed  to  consist  of  two  balanced,  yet  opposing,  forces: 
a  desire  to  stay  close  to  the  flock  and  a  desire  to  avoid  collisions  within  the  flock. 

Foraging  is  the  mechanism  by  which  ants  gather  food.  Over  the  years,  ants  have 
adopted  a  foraging  process  which  is  optimal  through  the  use  of  simple  commimications 
(in  the  form  of  chemical  hormone  deposits).  The  behavior  of  the  ants  in  this  process 
is  characteristic  of  swarm  behavior  in  that  simple  agents  are  able  to  work  together  and 
achieve  greater  accomplishments  as  a  group  than  as  a  number  of  individuals.  According  to 
the  Ant  Colony  Optimization  Home  Page  [6],  the  ants  lay  down  pheromone  trails  (chemical 
hormones)  when  food  is  discovered  and  carried  back  to  the  nest.  The  amount  of  pheromone 
dispensed  decreases  as  the  nest  is  approached  so  that  a  natmral  gradient  of  pheromone 
density  indicates  the  direction  of  the  food  somce.  The  more  ants  involved  in  the  harvesting 
of  the  food,  the  more  chemical  is  deposited.  The  simple  behavior  of  the  ants  is  to  move 
toward  increetsing  pheromone.  Over  a  short  period  of  time,  the  path  from  the  nest  to  the 
source  of  food  becomes  optimal  by  these  two  simple  processes  (depositing  pheromones  and 
moving  in  the  direction  of  higher  pheromones). 

These  processes  are  simple  examples  of  how  groups  of  animals  working  together  are 
able  to  achieve  more  than  animals  working  alone.  Some  of  these  basic  principals  have 
been  applied  to  multiple  agents  in  the  field  of  robotics  to  achieve  some  of  the  same  group 
dynamics  exhibited  by  natural  organisms. 

1.2.2  Robotics.  Scientists  in  the  field  of  robotics  have  developed  architec¬ 
tures  for  the  controlling  of  individual  robots  or  agents,  which  allow  groups  of  individuals 
to  experience  the  benefits  of  group  or  swarm  behaviors.  There  are  basically  two  ways  in 
robotics  to  communicate  information  between  agents.  One  is  to  have  each  communicate 
locally  to  its  neighbors,  and,  therefore,  the  receipt  of  information  is  limited  to  within  a 
certain  communication  range.  The  other  is  to  have  a  centralized  station  that  collects  all 
information  from  individual  agents  and  transmits  to  all  individual  agents.  This  latter 
method  is  less  enticing  to  the  munition  application  because  of  the  increased  reliability  of 
the  “dumb”  munitions  on  the  one  “smart”  station,  making  that  station  a  higher  priority 
target  for  the  enemy.  The  value  of  the  first  method  is  that  its  effectiveness  does  not  rely 
on  a  single  point  of  contact. 
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Arkin  studied  an  approach  to  cooperation  among  multiple  mobile  robots  without 
communications  [1].  In  this  study  a  multi-agent  simulation  was  used  to  emulate  the  be¬ 
havior  of  ant  foraging  by  having  robots  retrieve  objects  in  a  hostile  environment.  The 
robots  had  simple  behavioral  rides  that  they  were  required  to  follow,  and  they  emitted  sig¬ 
nals  with  different  gains  in  a  manner  very  similar  to  the  depositing  of  pheromones  by  ants. 
Arkin’s  research  showed  that  swarm  behavior  characteristics  could  be  achieved  without  any 
explicit  communication  between  the  agents.  He  also  showed  that  centralized  master/slave 
or  hierarchy-based  approaches  have  many  disadvantages  including  communication  bottle¬ 
necks,  less  robustness,  and  increased  complexity  when  compared  to  completely  decentral¬ 
ized  approaches  where  each  agent  follows  the  same  laws  and  behaviors  autonomously.  Kube 
and  Zhang  also  researched  the  use  of  decentralized  robots  performing  various  tasks  without 
explicit  communication  [10].  Much  of  their  research  was  comparing  the  applications  of  dif¬ 
ferent  social  insect  behaviors  to  robots  accomplishing  tasks.  In  another  contributing  work 
to  the  field,  Asama  sums  up  the  challenge  in  choosing  the  right  behaviors  for  your  agents 
by  saying  that  ‘‘an  autonomous  and  decentralized  system  has  two  essentially  contradictory 
characteristics,  autonomy  and  cooperativeness,  and  the  biggest  problem  in  the  study  of 
distibuted  autonomous  robotic  systems  is  how  to  reconcile  these  two  features”  [2]. 

Kwok  considered  the  problem  of  causing  multiple  (lOO’s)  of  autonomous  mobile 
robots  to  converge  to  a  target  using  an  on-board,  limited  range  sensor  for  sensing  the 
target  and  a  larger  but  also  limited- range  robot- to- robot  communication  capability  [11]. 
One  of  his  goals  was  to  keep  the  logic  as  simple  as  possible  aboard  each  robot,  so  he  imple¬ 
mented  a  “follow-the-leader”  approach  where  each  robot  followed  the  robot  with  the  best 
quality  of  information  concerning  the  location  of  the  target. 

While  much  of  the  research  in  the  field  of  cooperative  control  of  robotics  has  been  able 
to  apply  some  of  the  basic  principals  learned  from  ethology,  the  application  to  cooperative 
engagement  with  autonomous  weapons  is  rather  limited.  Since  each  of  the  munitions  has 
a  specific  Field  of  View  (FOV)  on  the  order  of  a  half  mile  in  width,  the  munitions  are 
normally  programmed  to  fly  a  half  mile  from  each  other  in  order  to  limit  the  FOV  overlap. 
Scenarios  exist  where  large  FOV  overlap  is  desired  in  the  interest  of  redundant  coverage 
and  higher  probabilities  of  success,  but  the  study  of  these  scenarios  is  more  applicable  to 
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the  cooperative  search  problem  than  the  cooperative  engagement  problem.  Therefore,  the 
protection  and  drag  efficiencies  gained  by  flocking,  schooling  or  herding  are  not  applicable 
to  this  study.  However,  the  concept  of  ant  foraging  does  have  application  to  the  problem 
at  hand.  Moreover,  what  if  the  ants  had  the  ability  to  choose  to  follow  the  pheromone 
deposits  to  the  known  source  of  food  or  to  choose  to  seek  out  a  different  area  for  a  possible 
larger,  better,  or  closer  food  source?  By  what  criteria  could  this  decision  be  made?  Is 
the  decision  criteria  the  same  for  all  situations?  Talcing  this  analogy  one  step  further  (and 
maybe  a  little  beyond  reality),  what  happens  when  an  ant  falsely  identifies  a  poisonous 
food  source  as  a  good  food  source  and  causes  the  colony  to  subsist  off  of  this  unknown 
danger?  These  questions  have  not  been  answered  in  the  applied  research  of  robotics  but 
are  extremely  important  questions  for  the  application  of  cooperative  control  of  autonomous 
wide  area  search  munitions. 


1.3  Objectives 

The  primary  objective  of  this  study  was  to  investigate  the  use  of  cooperative  behavior 
to  improve  the  overall  mission  effectiveness  of  autonomous  wide  area  search  munitions.  The 
specific  objectives  were  to: 

1.  Establish  a  methodology  for  measuring  the  expected  effectiveness  of  a  cooperative 
system  of  wide  area  search  munitions 

2.  Develop  optimal  cooperative  engagement  decision  rides  for  a  variety  of  realistic  sce¬ 
narios 

3.  Analyze  the  sensitivities  of  the  decision  rule  parameters  to  the  precision  of  the  sub- 
mxmition’s  ATR  algorithm,  the  lethality  of  the  warhead,  and  the  characteristics  of 
the  battlefield  (clutter  density,  target  layout,  etc.) 

1.4  Approach 

A  computer  simulation  was  used  to  emulate  the  behavior  of  autonomous  wide  area 
search  mimitions  and  measure  their  overall  expected  performance.  This  code  was  mod¬ 
ified  to  incorporate  the  capability  for  cooperative  engagement  based  on  a  parameterized 
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decision  rule.  Using  a  number  of  Design  of  Experiments  (DOE)  and  Response  Surface 
Methodologies  (RSM),  the  simulation  was  run  to  achieve  optimal  decision  rule  parameters 
for  given  scenarios  and  to  determine  the  sensitivities  of  those  parameters  to  the  precision 
of  the  ATR  algorithm,  lethality  of  the  warhead,  and  the  characteristics  of  the  battlefield. 

1,5  Scope 

This  research  is  not  limited  to  any  particular  type  of  wide  area  search  munition 
and  was  consciously  completed  using  parameters  that  describe  a  very  generic  wide  area 
search  munition.  This  research,  therefore,  applies  to  all  wide  area  search  munitions  and 
more  specific  results  can  be  achieved  for  any  specific  system  by  simply  modifying  the 
parameters  in  the  effectiveness  simulation.  Further,  the  methods  developed  as  part  of  this 
research  have  applications  in  the  more  general  area  of  cooperative  behavior  and  control. 
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II.  Wide  Area  Search  Munitions 


2.1  General  Characteristics  and  Operations 

According  to  Jacques  [9],  autonomous  wide  area  search  munitions  show  great  promise 
in  being  able  to  locate  and  engage  widely  dispersed  and/or  highly  mobile  or  relocatable 
groimd  targets.  They  have  the  effect  of  decentralizing  the  search  process  from  the  strike 
aircraft  or  smrveillance  sensors  to  greater  numbers  of  small,  smart  mrmitions  with  high  res¬ 
olution  seekers  operating  at  relatively  short  ranges.  Wide  area  search  munitions  equipped 
with  ATR  algorithms  can  be  delivered  with  very  relaxed  TLE  requirements  due  to  their 
ability  to  search  large  areas  and  make  autonomous  target  attack  decisions. 

In  measuring  the  effectiveness  of  wide  area  search  mmiitions,  Jacques  [9]  defines 
two  metrics  as  the  most  crucial  measures  of  ATR  performeince:  False  Target  Attack  Rate 
(FTAR)  and  probability  of  target  report  (Ptr)-  FTAR  is  defined  as  the  average  rate 
(/km^)  at  which  munitions  are  expended  on  falsely  confirmed  targets.  The  FTAR  is  driven 
by  the  target  being  searched  for,  the  environment  being  searched,  and  the  type  of  seeker 
and  ATR  algorithm  being  used  to  search.  The  numerical  expression  to  define  FTAR  is 
shown  in  equation  (2.1).  Ptr  is  the  probability  that  a  correct  attack  decision  is  made  given 
that  a  real  target  is  encountered.  The  expression  for  this  pareimeter  is  shown  in  equation 
(2.2).  Note  that  the  degree  of  discrimination  on  the  part  of  the  mimition  seeker  does  not 
have  to  be  specified. 


FTAR  —  7y  •  P fta\fte 


(2.1) 


where 


V 

Pfta\fte 


Clutter  Density 

Probability  of  false  target  attack  given  a  false  target  encounter 


PtR  =  Pocg  •  P/D 


(2.2) 
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where 


=  Probability  of  acquisition 

P/£)  =  Probability  of  correct  identification  (or  classification) 


A  generic  wide  area  search  munition  is  shown  in  Figure  2.1.  Some  parameters  that 
describe  the  wide  area  search  munition  used  in  this  research  are: 


•  Reliability  <  1 

•  False  Target  Atteick  Rate  >  0 

•  Probability  of  Target  Report  <  1 

•  Probability  of  Kill  <  1 

•  Maneuver  Capability  «  2  g’s 

•  Guidance  -  GPS/INS 


Vehicle  speed  -  lOOmIsec 


There  are  many  different  conceptual  search  patterns  for  wide  area  search  munitions 
including  line,  circle,  spiral,  and  serpentine.  The  pattern  chosen  for  this  study  was  the 
serpentine  pattern  demonstrated  by  four  munitions  in  Figure  2.2. 

The  notional  attack  operations  concept  for  wide  area  search  munitions  is  illustrated 
in  Figure  2.3.  The  concept  demonstrated  here  is  the  deployment  of  the  munitions  in 
an  attempt  to  find  a  relocatable  missile  laimcher.  The  event  that  triggers  this  entire 
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Figure  2.2  Serpentine  Search  Pattern 


operation  is  a  missile  launch.  Reconnaissance  satellites  detect  the  launch  and  transmit 
estimated  coordinates  of  the  launcher  to  a  C^I  aircraft.  That  aircraft  then  communicates 
the  information  to  the  ground  and  any  aircraft  in  the  area  in  an  attempt  to  task  resources 
to  attack  the  target.  A  combat  air  patrol  aircraft  carrying  the  wide  area  search  munitions 
is  then  notified  of  the  target.  That  aircraft  ingresses  to  the  target  location  and  deploys 
the  search  munitions.  As  the  figure  shows,  all  of  these  events  take  time,  and  the  area  of 
imcertainty  containing  the  missile  launcher  increases  quadratically  with  respect  to  time 
(assuming  the  target  is  free  to  move  in  any  direction).  In  order  to  overcome  this  increase 
in  uncertainty  area  for  the  single  target  in  this  example,  at  least  four  submunitions  must 
be  deployed  to  cover  the  required  area. 


2.2  ATR  Algorithm 

How  a  particular  ATR  works  in  reality  is  very  dependent  on  the  system  and  dis¬ 
crimination  level.  However,  for  purposes  of  effectiveness  analysis,  this  information  is  not 
required.  According  to  Jacques  [8],  the  typical  means  for  describing  the  ability  of  an  ATR 
based  system  to  make  correct  decisions  is  the  confusion  matrix.  To  start,  consider  the 
simplest  confusion  matrix  where  the  only  discrimination  is  between  target  and  non-target. 
The  confusion  matrix  for  this  simple  case  is  shown  in  Table  2.1.  In  a  simulation,  the  num¬ 
bers  in  the  confusion  matrix  are  used  to  determine  the  outcome  of  a  random  draw  each 
time  an  object,  target  or  otherwise,  is  encountered.  As  usual,  the  probability  numbers  in 
the  matrix  can  vary  between  zero  and  one,  with  a  perfect  algorithm  having  a  value  of  one 
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for  Ftr  and  a  value  of  zero  for  Ffta\fte-  Note  that  since  an  encountered  target  must  be 
declared  either  a  target  or  a  non-target,  the  sum  of  the  probabilities  in  any  row  must  sum 
to  one. 


Table  2.1  Binary  Confusion  Matrix 


Encountered 

Declared  As: 

Target  Non- Target 

Target 

Pth 

(1-Pth) 

Non-  Target 

^FTA\FTE 

(l-P  fta\fte) 

In  this  reseeirch,  two  real-target  types,  a  non-target  type,  and  clutter  were  used 
resulting  in  a  more  complex  confusion  matrix  shown  in  Table  2.2.  Target  1  represents  a 
high  priority  target  and  Target  2  represents  a  low  priority  target.  As  stated  previously,  all 
rows  sum  to  one  since  all  encountered  objects  must  be  declared  to  be  something.  Notice 
that  this  confusion  matrix  takes  into  account  the  possible  situation  that  a  non-target  more 
closely  resembles  a  low  priority  target  such  as  a  bus  or  cargo  truck  than  a  high  priority 
target  such  as  a  missile  launcher  or  tank. 

Table  2.2  Implemented  Confusion  Matrix 


Encountered 

Target  1 

Declared  As: 
Target  2  Non-Target 

Clutter 

Target  1 

Pth 

I-Pta 

3 

1-Pta 

3 

1— Pr« 

3 

Target  2 

1— PxR 

4 

Pth 

1-Pta 

2 

I-Pta 

4 

Non-Target 

I-Ptr 

4 

l-PrR 

2 

Pth 

1-Pth 

4 

Clutter 

P  FTA\FTE 

2 

P  FTA\FTE 

2 

0 

1-PfT/1|FTH 

The  two  seeker  parameters  this  study  is  most  interested  in  are  FTAR  and  PtR:  Ike 
latter  of  which  is  specifically  identified  in  the  confusion  matrix.  The  FTAR  is  computed 
by  the  expression  shown  in  equation  (2.3).  The  first  part  of  the  expression  represents  the 
incorrect  identification  of  clutter  as  a  real  target,  and  the  second  part  of  the  expression 
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represents  identifying  a  non-target  as  a  real  target  (both  situations  contributing  to  an 
overall  FTAR). 


FTAR  =  T]  •  p ft A\FTE  + 


#Non  Targets 
Battlefield  Area 


■( 


1  -  Ptr 
4 


+ 


1  - 


(2.3) 


2.3  Non- Cooperating  Sensitivities 

According  to  the  wide  area  search  munition  effectiveness  analysis  completed  by 
Jacques  [9],  the  performance  of  non-cooperating  munitions  is  extremely  sensitive  to  FTAR. 
Figures  2.4  and  2.5  illustrate  this  sensitivity  for  a  very  simple  sceneurio.  The  scenario  used 
weis  that  which  was  described  in  Figure  2.3,  that  of  a  single  mobile  missile  launcher  fleeing 
its  laimdi  location.  The  different  lines  in  Figure  2.4  represent  the  size  of  the  search  area 
required  to  guaramtee  an  encounter  with  the  target.  The  nmnber  of  munitions  required  to 
achieve  an  expected  mission  success  greater  than  80%  is  plotted  versus  a  varying  FTAR. 
It  is  easy  to  see  the  extreme  sensitivity  to  an  increasing  FTAR. 

Number  of  Munitions  Required  To  Cover  Specified  Area 
Required  P  Mission  Success  >  .8 


— *-25km'2 
-♦-50  km'2 
. lOOkm^Z 


Faise  Target  Attack  Rate  (/km*2) 


Figure  2.4  Munition  ^’s  vs.  FTAR  and  Area  Coverage 
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Figure  2.5  once  again  shows  the  sensitivity  to  FTAR,  but  this  time  parameterized 
around  Ptr  for  a  given  level  of  mission  success  required  (80%)  and  a  given  search  area 
(50  km^).  It  is  interesting  to  note  that  the  single  fleeing  target  scenario  is  relatively 
insensitive  to  Ptr  when  compared  to  the  sensitivity  to  FTAR. 


Munitions  Req'd  vs.  False  Target  Attack  Rate 
Required  P  Mission  Success  =  0.8,  A  =  50  km^2 


0.001  0.003  0.010  0.032  0.100  0.316  1.000  3.162  10.000 


False  Target  Attack  Rate  (/km'^2) 

Figure  2.5  Munition  #’s  vs.  FTAR  and  Probability  of  Target  Report 

2.4  Effectiveness  Prediction  Methods 

When  predicting  the  effectiveness  of  wide  area  search  munitions  with  the  inclusion  of 
these  two  key  parameters  (FTAR  and  Ptr)  in  a  multi-target/multi- munition  scenario,  an¬ 
alytic  or  deterministic  methods  are  simply  infeasible.  For  the  single  target,  single  mimition 
case,  a  Pk  could  be  determined  by  equation  (2.4). 

Pk\bnc  —  Plos  •  Ptr  •  P h\tr  '  Pk\h  (2-4) 

where 

Pk\enc  —  Probability  of  kill  given  a  real  target  encounter 
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Plos  =  Probability  of  having  a  clear  line  of  sight 
Ph\tr  =  Probability  of  hitting  the  target  given  that  you  found  it  and 
correctly  recognized  it  (primarily  a  guidance  parameter) 

Pk\h  =  Probability  of  killing  the  target  given  you  hit  it  (a  warhead 
parameter) 

Although  not  shown,  equation  (2.4)  could  be  generalized  to  accommodate  reliability  fac¬ 
tors,  and  this  typically  is  done  for  most  effeciveness  models. 

However,  given  that  the  scenario  is  not  that  simple,  how  does  one  deterministically 
model  a  group  of  munitions  wading  through  a  number  of  non-targets  and/or  clutter  on 
their  way  to  a  number  of  real  targets  that  they  might  not  be  able  to  correctly  identify  when 
they  get  there?  Then  the  addition  of  cooperative  behavior  among  the  mimitions  would 
make  any  euialytic  solutions  simply  impossible.  Therefore,  an  appropriate  method  for 
measuring  the  effectiveness  of  wide  area  search  munitions  is  the  use  of  a  stochastic,  Monte 
Carlo  based  computer  simulation.  The  computer  simulation  allows  the  incorporation  of 
all  desired  cooperative  behaviors,  the  setting  of  desired  probabilities  and  FTARs,  and  the 
measurement  of  expected  effectiveness  values.  Having  the  simulation  Monte  Carlo  based 
provides  a  means  for  measuring  the  expected  performance  over  many  random  probability 
draws.  This  was  the  method  chosen  to  accomplish  all  effectiveness  analysis  in  this  research. 
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III.  The  Computer  Simulation 

3.1  Baseline  Simulation 

The  Monte  Carlo  based  Fortran  program  employed  by  this  research  was  originally 
developed  by  Lockheed  Martin  Vought  Systems  [12]  as  an  effectiveness  model  for  the 
LOCAAS.  However,  it  is  versatile  enough  to  be  used  for  any  generic  wide  area  search 
munition.  The  simulation  makes  no  attempt  to  model  the  aerodynamics,  guidance,  etc.  of 
the  submunitions,  however,  it  does  model  multiple  submunitions  in  a  coordinated  search 
for  multiple  teirgets.  Prior  to  the  modifications  made  through  this  research,  this  program 
had  the  capability  to  simulate  the  following  events  of  the  submunition  “life  cycle” : 

•  Round  dispense  (any  number  of  rovmds) 

•  Submunition  dispense  (any  number  of  submxmitions  per  roimd) 

•  Submunition  flies  a  user  supplied  pattern  by  following  predetermined  waypoints  and 
looks  for  tcirgets  on  the  groimd 

•  If  a  target  enters  a  submunition’s  FOV,  the  submunition  may  acquire  it  based  on  the 
precision  of  the  ATR  eJgorithm 

•  Once  acquired,  the  submunition  can  select  that  target  to  engage 

•  Once  engaged,  the  submunition  attempts  to  hit  the  target 

•  Once  the  target  is  hit,  an  assessment  is  made  as  to  whether  the  target  has  been 
completely  destroyed  (dead)  or  is  stiU  in  working  condition  (alive) 

The  simulation  allows  for  any  nmnber  of  targets  with  varying  priority  levels,  the  ad¬ 
dition  of  non-targets  (military  or  civilian),  and  a  user  supplied  clutter  density  per  square 
kilometer  of  battlefield.  The  baseline  code  also  has  an  option  to  incorporate  some  cooper¬ 
ation  among  the  submunitions.  This  option  has  the  following  limited  capabilities: 

•  When  a  target  is  engaged  by  a  submunition,  send  just  one,  £dl,  5,  etc.  of  the 
remaining  submunitions  to  engage  that  target 
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•  Once  called  to  cooperate,  the  submunitions  could  go  immediately  or  at  the  end  of 
the  current  search  leg 

When  a  submunition  is  redirected  oflF  of  its  initial  user  supplied  flight  path  (due  to  a 
cooperative  engagement  attempt)  but  does  not  see  the  target  it  is  supposed  to  engage, 
that  submimition  will  fly  a  straight  line  for  twenty  seconds  beyond  the  expected  target 
location  and  then  expire.  The  submimition  might  not  see  the  target  it’s  redirected  toward 
for  a  number  of  possible  reasons: 

1.  The  submunition’s  detection  mechanism  does  not  acquire  the  target 

2.  The  submunition  acquires  the  target  but  falsely  identifies  it  cts  either  clutter  or  a 
non-target  and,  therefore,  does  not  select  (engage)  that  target 

3.  The  submunition  that  originally  engages  the  target  falsely  identified  a  non-target 
or  clutter  as  a  real  target  and  commimicates  the  position  of  this  object  as  a  real 
target  to  the  other  submunitions;  consequently,  the  submunition  that  is  redirected 
as  a  result  of  that  information  is  sent  to  an  area  where  there  are  no  real  targets 

The  simulation  is  extremely  flexible  in  its  capabilities  to  handle  a  multitude  of  input 
parameters  and  supplies  comprehensive  results  as  output  files  at  the  conclusion  of  each 
run. 


3.1  A  Inputs  to  the  Simulation.  To  nm  the  simulation,  two  separate 
input  files  are  required.  The  first  contains  the  information  concerning  the  user  supplied 
flight  paths  for  the  submimitions  once  dispensed  from  the  roxmds  including  waypoints, 
altitude  and  velocity  (see  appendix  A,  page  A-5  for  an  example).  The  second  input  file 
contains  all  the  parameters  characterizing  the  submunitions  and  the  parameters  required  to 
nm  the  simulation  (see  appendix  A,  page  A-1  for  an  example).  Table  3.1  shows  a  summary 
of  some  of  the  input  parameters  that  must  be  entered  regarding  the  characteristics  of  the 
submunitions  and  targets: 

Since  the  simulation  is  based  on  the  ability  to  do  Monte  Carlo  runs,  the  user  also  has 
the  ability  to  pick  the  number  of  Monte  Carlo  trials  and  a  Monte  Carlo  baseline  random 
number  stream  seed  (which  is  modified  for  every  repetition  in  a  series). 
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Table  3.1  Input  Parameters  to  Baseline  Simulation 


Parameter _ 

Numbers: 

Rounds 
Submunitions 
Target  Types 
Targets 
Discrete: 

Random  Targets 

Blind  in  Turns 
Live 

Correlate  Targets 
Reliabilities: 

Roimd 

Submimition 

Probabilities: 

Acquisition 

Hit 

Kill 

Correct  Identification 

Seeker  Data: 

Foot  Print  Width 
Beam  Width 
Boresight  Angle 
Scan  Time 
Flyback  Time 
Submunition  Data: 
Min  Turn  Radius 
Time  of  Flight 
Target  Data: 
Locations 
Mobility  Data 


Description 


Total  number  of  rounds  dispensed 

Total  number  of  submunitions  (and  submunitions  per  round 
Priority  1,  priority  2,  non-targets,  etc. 

Total  number  of  targets  and  how  many  of  each  type 

Either  targets  are  placed  in  specific  locations  or  random  within  a 
specified  area  <  total  battlefield  area 
Submunition’s  target  detection  is  turned  off  when  turning 
Submunition  can  tell  if  target  was  killed  by  another  submunition  already 
Submunition  can  tell  if  it  has  seen  a  target  before 

Probability  that  roimd  will  not  fail 
Probability  that  submunition  will  not  fail 
(Input  for  each  target  type) 

Submunition  will  acquire  the  target  when  it  enters  its  FOV 

Submunition  will  hit  the  target  once  its  acquired 

Submunition  will  kill  the  target  once  its  hit 

Submunition  will  identify  the  target  correctly  or  incorrectly 

(incorrect  identifications  are  distributed  among  adl  target  types  as  desired) 

Width  of  the  FOV  on  the  ground 

Beam  width  in  degrees  used  for  vertical  FOV 

Angle  at  which  the  LADAR  points  down  from  the  horizon 

Time  for  the  FOV  to  sweep  the  entire  foot  print  width 

Time  for  the  FOV  to  return  at  the  completion  of  each  sweep 

Minimum  turn  rcidius  the  submunition  can  fly 

Total  Time  of  flight  from  submunition  dispense  time  to  expiration 

Specific  locations  of  all  targets  if  using  non-random  target  layout 
If  mobile:  start  time,  heading,  speed,  acceleration  time 
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3.1.2  Outputs  of  the  Simulation.  The  simulation  offers  four  different 
output  files.  The  main  output  file  first  lists  all  of  the  input  parameters  used  to  run  the 
simulation  for  tracking  purposes.  Then  for  each  Monte  Carlo  repetition,  a  brief  history 
of  what  each  submunition  did  during  that  repetition  is  displayed.  Finally,  at  the  end  of 
the  main  output  file,  all  Monte  Carlo  repetitions  eire  summarized  showing  a  breakdown, 
per  target  type  and  per  individual  target,  of  the  number  of  acquisitions,  selections,  hits, 
and  kills,  as  well  as  the  total  number  of  kills  and  unique  kills  for  that  simulation  run.  An 
example  of  this  output  file  for  10  Monte  Carlo  runs  (as  opposed  to  200  used  for  all  testing) 
is  located  in  appendix  A  on  page  A-18. 

The  following  files  can  be  created  for  any  number  of  individual  repetitions  within  a 
Monte  Carlo  cycle.  The  history  output  file  recaps  everything  that  happened  dmring  the 
repetition  in  chronological  order  in  a  manner  similar  to  that  of  a  sports  announcer  providing 
play  by  play  eiction  (see  appendix  A,  page  A-29  for  an  example).  The  next  output  file  is 
a  summary  of  the  cuts  for  each  submunition.  A  cut  occurs  anytime  a  submunition  sees 
something  on  the  ground  and  the  ATR  algorithm  takes  a  “cut”  at  classifying  it.  The 
cuts  output  file  is  broken  down  by  submunition,  and  provides  specific  information  about 
each  cut  that  submimition  had  during  that  repetition  (see  appendix  A,  page  A-31  for  an 
example).  Finally,  the  playback  output  file  is  a  data  file  for  a  Visual  Basic  program  that 
provides  a  post-processed  visual  representation  of  the  entire  repetition  (see  appendix  A, 
page  A-32  for  a  sample  screen  capture). 

3.2  Simulation  Modifications 

Modification  of  the  simulation  was  accomplished  in  three  steps: 

•  Define  and  provide  shared  information  for  use  by  the  individual  submunitions 

•  Implement  the  cooperative  engagement  decision  algorithm 

•  Implement  other  small  changes  to  best  Eichieve  all  objectives 

Once  the  code  was  pretty  well  understood,  the  following  modifications  were  made. 
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3,2.1  Shared  Information.  The  first  step  in  the  modification  process  was 
to  be  able  to  redirect  any  number  of  submimitions  at  ciny  time  toward  any  foimd  targets. 
The  way  this  was  accomplished  was  using  a  structiued  array  to  store  all  target  information 
on  the  targets  found.  This  structured  array  stored  the  x,  y  and  z  coordinates  of  the  target 
(if  the  target  was  a  moving  target,  these  coordinates  would  be  those  corresponding  to  the 
position  of  the  target  at  the  time  it  was  acquired  eind  selected),  and  the  type  of  target 
found.  A  very  important  distinction  which  needs  to  be  made  at  this  point  is  that  the 
target  type  stored  is  not  necessarily  the  correct  identification  of  the  target  found;  it  is  the 
identification  of  the  target  type  determined  by  the  munition  that  identified  that  target. 
Therefore,  the  type  of  target  foimd  which  is  stored  in  this  tjirget  array  may  not  be  the  true 
type  of  target  located  at  the  stored  coordinates. 

Once  the  target  information  was  stored,  a  method  for  distributing  that  information 
had  to  be  determined.  Obviously,  since  this  was  just  a  simulation,  it  would  be  easy  to 
just  provide  all  submimitions  access  to  all  entries  in  the  ciforementioned  structured  ar¬ 
ray.  But  is  this  feasible,  realistic  or  even  advantageous?  Since  this  study  hoped  to  gain 
some  insight  into  the  trade-offs  between  loccd  emd  global  communication,  a  mechanism 
for  determining  whether  a  submunition  received  the  communicated  information  had  to  be 
implemented.  First  of  all,  in  this  study  incomplete  communications  were  not  considered, 
i.e.,  either  a  submunition  receives  all  the  information  about  the  target  or  none.  However, 
communications  reliability  based  solely  on  whether  or  not  the  submunition  was  within  a 
certain  maximum  communications  range  did  not  seem  too  realistic  either.  Therefore,  a 
communications  reliability  function  was  developed.  In  order  to  keep  it  relatively  simple, 
this  function  was  based  solely  on  the  probability  of  communication  failures  increasing  as 
maximum  communications  range  was  approached.  The  function  used  is  shown  in  equation 
3.1.  Another  way  of  doing  this  (and  possibly  a  more  realistic  way)  could  have  been  to 
assign  50%  reliability  (possibly  representing  a  3dB  degradation  in  signal  power)  at  the 
maximum  communications  range  and  then  continue  the  function  down  to  0%  reliability 
some  distance  beyond  the  maximum  communications  range.  This  would  make  the  maxi¬ 
mum  communications  range  more  of  a  “soft”  constraint,  allowing  some  communications  to 
be  received  beyond  that  distance,  rather  than  a  “hard”  constraint  as  done  in  this  study. 
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Comm  Rel  = 
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( 


max  comm  range- range  ^ 


max  comm  range 


0,1 


•  r  ^  max  comm  range 

if  range  < - 2 - ^ 

it  range  >HfiL£2mSLEffiE 


(3.1) 


Figure  3.1  illustrates  an  example  of  this  communications  reliability  function  for  a  maximum 
communications  range  of  10,000  meters. 


Figure  3.1  Communications  Reliability  Function 


In  order  to  implement  the  decision  algorithm  described  in  the  following  section, 
the  amount  of  information  that  had  to  be  shared  among  the  submunitions  had  to  be 
determined.  For  this  study,  the  following  three  pieces  of  information  were  determined  to 
be  importcint  and  were  communicated  for  each  tcirget  foimd 


•  Location  of  the  target 

•  Type  of  target 

•  Specific  target  to  be  engaged 

The  location  of  the  target  was  communicated  £is  the  precise  x  and  y  coordinates  of  the 
target.  The  type  of  target  was  communicated  as  either  a  high  priority  (priority  1)  or  a 
low  priority  (priority  2).  Once  again,  this  communicated  target  type  is  the  target  type  as 
identified  by  the  ATR  zilgorithm  eind,  therefore,  not  necesscirily  a  correct  classification  (or 
priority  identification)  of  the  target.  The  specific  target  to  be  engaged  is,  in  reality,  a  very 
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difiicult  piece  of  information  to  communicate  and  keep  track  of  reliably,  especially  with 
non-global  emd  non-perfect  commimications.  In  particular,  multiple  targets  within  a  small 
area  can  cause  ambiguities  as  to  which  target  is  really  being  seen.  However,  in  this  study, 
the  target  registration  problem  was  not  considered. 

3.2.2  Decision  Algorithm.  The  purpose  of  the  decision  algorithm  was  to 
provide  a  criteria  by  which  the  submimitions  could  “decide”  whether  or  not  to  participate 
in  a  cooperative  engagement.  In  developing  the  algorithm,  the  goals  were  to  incorporate  all 
important  factors  that  should  be  tciken  into  accoimt  for  making  a  cooperative  engagement 
decision  and  to  keep  it  simple  since  the  available  computing  power  aboard  these  submu¬ 
nitions  is  likely  to  be  minimal.  After  many  long  braunstorming  sessions,  the  following  (in 
no  particular  order)  were  determined  to  be  the  most  important  factors  that  needed  to  be 
included  in  the  decision  algorithm; 

•  Fuel  Remaining 

•  Target  Priority 

•  Range  Rate  from  submunition  to  target 

•  Range  from  submunition  to  target 

•  Number  of  submunitions  that  have  already  engaged  a  particular  target 

To  keep  the  decision  algorithm  simple,  the  basic  first  order  expression  shown  in 
equation  3.2  was  used. 

Threshold  =  ai  *  si  +  a2  *  2:2  -I-  as  *  xa  —  04  *  X4.  (3.2) 


where 

xi  =  Normalized  Fuel  Remaining 
X2  =  Normalized  Target  Priority 
X3  =  Normalized  Range  Rate 

X4  =  Normalized  Number  of  Engaged  Submunitions  on  a  particular  target 
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Weighting  Parameters 


ai  = 


Note  that  the  actual  range  from  the  submunition  to  a  specific  target  is  not  explicitly 
used  in  this  decision  algorithm,  however,  a  range  check  was  added  as  a  final  go/no-go 
criteria  to  ensure  that  a  submunition  is  not  redirected  toward  a  target  that  cannot  be 
reached  based  on  insufiicient  fuel  remaining. 

All  factors  in  the  decision  algorithm  were  normalized  with  the  sense  that  values 
approaching  imity  encouraged  a  cooperative  attack,  while  values  approaching  zero  provided 
a  discouragement.  One  exception  to  this  is  the  0:4  parameter  which  is  explained  below. 
Normalization  of  fuel  remaining  in  the  simulation  was  easily  accomplished  by  normalizing 
time  of  flight  or  search  time.  Since  each  submunition  was  assumed  to  have  a  twenty 
minute  total  search  time,  the  normalized  time  of  flight  was  the  time  in  search  divided  by 
1200  seconds.  Target  priority  was  normailized  by  assigning  a  value  of  one  to  a  priority  one 
target,  one-half  to  a  priority  2  target  and  zero  for  anything  else. 

The  piurpose  of  incorporating  a  range  rate  parameter  in  the  decision  rule  was  to  apply 
little  influence  on  the  decision  (or  even  discourage  a  cooperative  engagement)  when  the 
range  rate  was  negative  (the  submimition  is  moving  towards  the  target)  and  to  encourage 
a  cooperative  engagement  when  the  range  rate  is  positive  (the  submunition  is  moving 
away  from  the  target).  This  provided  a  means  for  allowing  the  submimition  to  continue 
its  predetermined  search  pattern  if  it  was  flying  toward  a  known  target  location.  The 
expression  used  to  normalize  range  rate  is  shown  in  equation  3.3  with  r  defined  by  a 
backward  difierence. 


where 


normalized  range  rate  = 


r  —  vel 
2  *  vel 


(3.3) 


rangCj  -  range^.j 

r  =  — : - : - 

timei  —  timei_i 

Figure  3.2  illustrates  the  function  for  normalized  range  rate  shown  in  equation  (3.3).  Note 
that  equation  (3.3)  and  Figure  3.2  correspond  to  an  assumed  mimition  speed  of  100  m/sec. 
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Figure  3.2  Normalized  Range  Rate 

Finally,  the  last  parameter  in  the  decision  algorithm  is  the  normalized  number  of 
engaged  submunitions  on  a  specific  target.  The  pmrpose  of  this  parameter  is  to  discourage 
multiple  cooperative  engagements  on  a  single  target  in  an  attempt  to  spread  out  the  total 
hits  and  not  send  all  submimitions  after  the  same  target.  When  a  target  has  been  engaged 
by  only  a  single  submunition,  then  this  parameter  should  not  be  discouraging  a  cooperative 
engagament  on  that  target.  However,  once  one  submunition  has  cooperatively  engaged  a 
target  (resulting  in  a  total  of  two  munitions  attempting  to  hit  that  specific  target),  this 
parameter  should  be  invoked  to  discomrage  any  additional  submimitions  from  choosing  to 
cooperatively  engage  that  target.  Equation  3.4  was  used  to  normalize  this  parameter. 

Normalized  Parameter  =  Number  of  engaged  submunitions  —  1  (3.4) 

Note  that  in  equation  3.2  a  sign  is  implemented  in  front  of  this  parameter  in  order 
to  discourage  a  cooperative  engagement  3is  this  parameter  increases.  As  desired,  this 
parameter  equals  zero  when  only  one  submunition  has  engaged  a  specific  target  but  then 
increases  in  value  as  more  submunitions  cooperatively  engage  that  target. 

The  implementation  of  the  decision  rule  in  equation  3.2  was  rather  simple.  Once  a 
target  is  found,  the  information  about  that  target  is  communicated  by  the  submunition  that 
identified  the  target.  Then  at  all  subsequent  time  steps,  every  submunition  that  received 
the  commimication  and  is  not  in  an  engaged  status  will  calculate  aJl  of  the  normalized  pa- 
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rameters  and  the  decision  algorithm.  When  multiple  targets  are  found  and  communicated, 
then  at  all  subsequent  time  steps  the  normalized  parameters  and  the  decision  algorithm 
are  calulated  by  each  submimition  for  each  target  individually.  If  the  total  for  the  decision 
algorithm  exceeds  the  decision  threshold,  then  a  cooperative  engagement  on  that  target 
by  that  submunition  occurs.  That  submunition  then  communicates  globally  with  100% 
reliability  that  it  has  engaged  that  specific  target  (ignoring  all  target  registration  issues). 

3.2.3  Additional  Modifications.  In  order  to  best  achieve  the  objectives 
of  this  study,  a  few  additional  changes  needed  to  be  made  to  the  simulation.  The  first 
was  a  simple  modification  to  the  main  output  file  to  include  the  values  of  each  normalized 
parameter  as  well  as  the  weights  on  the  parameters  every  time  a  cooperative  engagement 
was  invoked.  This  provided  a  means  to  track  all  cooperative  engagements,  and  ensure  the 
decision  algorithm  was  being  implemented  properly. 

A  second  change  was  an  attempt  to  answer  the  following  question:  what  should 
happen  to  a  submunition  that  is  sent  off  its  original  search  pattern  to  cooperatively  engage 
a  target  that  it  cannot  find?  Three  possible  scenarios  that  could  cause  this  situation  were 
listed  on  page  3-2.  This  issue  was  not  addressed  in  the  baseline  code.  In  order  to  resolve 
this,  an  attempt  was  made  to  create  a  new  search  pattern  for  the  redirected  submunition 
that  focused  on  the  location  of  the  target  that  was  cooperatively  engaged.  The  new  pattern 
used  was  a  growing  figure-8  centered  on  the  communicated  target  location.  This  pattern 
woidd  initially  turn  the  submunition  around  after  it  crossed  the  expected  target  location 
to  fly  right  back  over  it  as  an  attempt  to  acquire  and  classify  the  target  if  it  simply 
“missed”  it  the  first  time.  If  the  target  was  still  not  selected  on  this  second  pass,  then  the 
submunition  would  continue  flying  past  the  target,  but  this  time  farther  past  the  target, 
in  the  opposite  direction  in  which  the  submunition  first  approached  the  target  area.  It 
would  then  turnaround  and  fly  back  toward  the  target  imtil  the  submimition  engages  a 
target  or  expires  (search  time  depletes)-the  submunition  cannot  participate  in  a  second 
cooperative  engagement  on  a  different  target.  The  behavior  chosen  to  handle  this  situation 
is  not  necessarily  that  which  would  be  implemented  operationally,  nor  was  any  research 
completed  that  showed  this  behavior  would  produce  optimal  results.  This  situation  was 
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deemed  outside  the  scope  of  the  research  and  could  better  be  addressed  by  a  study  in 
cooperative  search. 
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IV.  Applied  RSM 

4 . 1  Introduction 

Response  surface  methodology  is  a  collection  of  statistical  and  mathematical  tech¬ 
niques  useful  for  developing,  improving,  and  optimizing  processes.  Most  applications  in¬ 
clude  situations  where  several  input  variables  potentially  influence  some  performance  mear 
sure  or  quality  characteristic  of  the  system.  The  purpose  of  RSM  is  to  approximate  the 
measures  of  performance,  referred  to  as  response  functions,  in  terms  of  the  most  critical 
factors  (or  independent  variables)  that  influence  those  responses.  In  doing  this,  a  response 
surface  can  then  be  mapped  out  (hypersurface  for  the  general  n-dimensional  case)  showing 
how  variations  in  the  independent  variables  ciflFect  the  responses. 

A  typical  RSM  according  to  Myers  and  Montgomery  [14]  is  broken  into  phases.  The 
first  experiment  is  usually  designed  to  investigate  which  factors  are  most  influential  to  the 
responses  with  the  intent  of  eliminating  the  imimportant  ones.  This  type  of  experiment 
is  called  a  screening  experiment  and  is  typically  referred  to  as  phase  zero  of  a  response 
siurface  study.  Once  the  important  independent  variables  are  identified,  phase  one  begins 
with  the  intent  of  determining  if  the  current  levels  or  settings  of  the  independent  variables 
result  in  values  of  the  responses  near  optimum.  If  the  current  settings  or  levels  of  the 
independent  variables  are  not  consistent  with  optimum  performance,  then  adjustments  to 
the  input  variables  that  will  move  the  responses  toward  the  optimum  must  be  made.  To 
do  this  a  first  order  model  and  the  optimization  technique  called  the  method  of  steepest 
ascent  are  employed.  Phase  two  of  the  response  surface  study  begins  when  the  process  is 
near  the  optimum.  At  this  point  models  that  will  accurately  approximate  the  true  response 
functions  are  desired.  Because  the  true  response  surfaces  usually  exhibits  curvature  near 
the  optimum,  models  of  at  least  second  order  must  be  used.  Finally,  the  models  of  the 
various  responses  must  be  analyzed  to  determine  the  settings  for  the  independent  variables 
that  provide  the  optimal  expected  performance  over  all  responses. 
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J^.2  Application 


For  this  study,  RSM  was  used  to  determine  the  optimal  settings  for  the  a^’s  in  the 
cooperative  engagement  decision  algorithm  shown  in  equation  3.2  as  well  as  the  maximum 
communications  range.  The  optimal  a^’s  were  simply  the  weights  on  the  parameters  used 
in  the  decision  rules.  A  high  weight  on  a  parameter  means  that  that  parameter  is  of 
high  importance  in  making  the  decision  to  cooperatively  engage,  whereas  a  low  weight  on 
a  parameter  can  be  interpreted  to  mean  that  that  parameter  is  less  important  (or  even 
insignificant)  in  making  the  decision  to  cooperatively  engage.  A  low  maximum  communica¬ 
tions  range  implies  that  only  local  communications  are  employed,  whereas  a  high  maximum 
communications  range  implies  global  communications. 

4*2.1  Independent  Variables.  Because  of  the  relatively  small  number  of 
input  variables,  the  phase  zero  screening  experiments  were  not  necessary  for  this  study. 
Therefore,  the  first  step  was  to  choose  the  ranges  of  the  independent  variables  to  begin 
the  RSM.  To  do  this  effectively,  a  decision  threshold  from  equation  3.2  had  to  first  be 
determined  and  fixed.  Without  loss  of  generality,  a  threshold  of  one  was  chosen  and 
remained  constant  for  ail  simulations.  In  picking  the  ranges  for  the  independent  variables, 
careful  consideration  was  made  to  ensure  the  RSM  studies  would  be  investigating  the 
effects  of  different  cooperative  engagement  decision  rules.  Therefore,  the  values  for  the 
independent  variables  when  chosen  at  their  extremes  had  to  be  able  to  result  in  the  possible 
triggering  of  the  decision  algorithm.  Since  the  first  three  parameters  in  the  decision  rule 
(as  described  on  page  3-7)  were  normalized  to  have  maximum  values  of  one,  the  weights 
on  these  parameters  could  not  be  all  less  than  one-third  or  else  a  cooperative  engagament 
would  be  impossible.  This  would,  therefore,  result  in  an  RSM  study  investigating  the 
effects  of  different  cooperative  engagament  decision  rules  and  no  cooperation  at  all.  Since 
this  was  not  the  goal,  minimum  values  for  the  first  three  parameters  had  to  be  chosen 
greater  than  one  third.  Table  4.1  shows  the  values  used  for  each  independent  variable. 

The  values  chosen  in  Table  4.1  ensure  that  even  when  the  independent  variables 
are  chosen  at  their  extremes,  cooperative  engagements  are  still  possible.  The  maximum 
commimications  range  values  were  chosen  based  on  a  battlefield  that  was  approximately 
300  square  kilometers  in  size. 
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Table  4.1  Independent  Variable  Ranges  for  RSM 


Variable 

Weight  on 

Minimum 

Maximum 

ai 

Time  of  Flight 

0.4 

0.8 

OL2 

Target  Priority 

0.4 

0.8 

«3 

Range  Rate 

0.4 

0.8 

OLA 

Number  of  Munitions 

0.4 

0.8 

Maximum  Communications  Range 

5  km 

15  km 

4>2.2  Responses.  The  responses  had  to  be  chosen  to  somehow  accurately 
measure  the  expected  mission  effectiveness  for  wide  area  search  munitions.  Four  responses 
were  chosen: 


•  Unique  Kills 

•  Total  Kills 

•  Total  Hits 

•  Target  Formula 

Unique  kills  was  defined  as  the  expected  number  of  vmique,  real  targets  killed  (each  target 
can  be  killed  only  once).  Toted  kills  was  defined  as  the  number  of  submunitions  expected 
to  achieve  lethal  hits  on  real  teirgets.  Total  hits  was  defined  as  the  expected  number  of 
real  target  hits.  Finally,  the  teurget  formula  response  was  used  as  a  means  of  measuring  the 
hits  on  high  priority  targets  (priority  one)  versus  hits  on  low  priority  targets  (priority  two) 
and  incorporating  a  penalty  for  any  hits  on  non-targets.  This  target  formula  is  shown  in 
equation  4.1  where  “#  prior  1  hits”  means  the  nrunber  of  hits  on  priority  one  targets,  “# 
prior  2  hits”  means  the  nmnber  of  hits  on  priority  two  targets,  and  non-target  hits” 
mecins  the  number  of  hits  on  any  non-targets. 


Target  Formula  =  2  *  (^  prior  1  hits)  -f  #  prior  2  hits  —  #  non-target  hits  (4.1) 

A  simple  example  can  be  used  to  distinguish  and  better  understand  these  responses. 
This  example  has  five  submunitions,  2  real  targets  (one  high  priority  and  one  low  priority) 
and  one  non- target.  Submunition  #1  hits  target  #1,  a  high  priority  target,  but  does  not 
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kill  it.  Submunition  #2  hits  that  same  target  (target  #1)  but  kills  it.  Submunition  #3 
hits  and  kills  target  #2,  a  low  priority  target.  Then  submunition  #4  also  hits  target  #2, 
and  this  engagement  is  also  deemed  a  kill  (even  though  the  target  was  already  dead)  . 
Finally,  submunition  #5  hits  the  non-target.  The  responses  for  this  example  £ire  shown  in 
Table  4.2. 


Table  4.2  Responses  for  Example 


Response 

Explanation 

Value 

Unique  Kills 

targets  #1  cUid  #2 

2 

Total  Kills 

submunitions  #2,  #3,  and  #4 

3 

Total  Hits 

submunitions  #1,  #2,  #3,  and  #4 

4 

Target  Formula 

2  hits  on  target  #1  (high  priority  target) 

2  hits  on  target  #2  (low  priority  target) 

1  hit  on  a  non-target 

5 

4>2.3  Phase  1.  The  purpose  of  phase  one  is  to  determine  if  the  current 
ranges  for  the  independent  variables  shown  in  Table  4.1  result  in  responses  that  eire  near 
optimal.  To  accomplish  this  a  fractional  factorial  design  was  used.  This  design  is 
both  orthogonal  and  resolution  V.  Orthogonality  is  often  a  desired  property  for  ease  and 
convenience  of  calculations  and  the  avoidance  of  singularities.  A  resolution  V  design  is 
a  design  in  which  no  main  effect  or  two-factor  interaction  is  aliased  with  any  other  main 
effect  or  two-factor  interaction,  but  two-factor  interactions  are  aliased  with  three-factor 
interactions.  In  general,  the  resolution  of  a  two-level  fractional  factorial  design  is  equal  to 
the  smallest  number  of  letters  in  any  word  in  the  defining  relation.  Usually,  the  fractional 
design  having  the  highest  possible  resolution  consistent  with  the  degree  of  fractionation 
required  is  desired.  This  provides  a  design  with  less  restrictive  assumptions  regarding 
which  interactions  are  negligible  in  order  to  obtain  a  imique  interpretation  of  the  data. 
This  property  is  most  desired  for  Phase  2  described  below  and  is  therefore  used  in  this 
phase  to  best  garner  the  benefits  of  sequential  testing.  This  design  required  a  total  of  16 
runs  to  complete.  Each  design  was  augmented  with  four  center  runs  resulting  in  a  total 
of  20  runs.  For  each  run  the  values  for  each  of  the  four  responses  were  recorded.  Using 
an  analysis  of  variance  (ANOVA)  for  each  response,  an  attempt  to  fit  first  order  models 
to  each  response  was  made.  Whenever  a  first  order  model  was  appropriate,  the  method  of 
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steepest  ascent  was  used  to  traverse  the  response  surface  to  a  new  operating  region  that 
was  closer  to  the  optimal  design  point.  The  method  of  steepest  ascent  is  summarized  by 
the  following  few  steps. 

1.  Fit  a  planar  (first-order)  model  using  an  orthogonal  design 

2.  Compute  a  path  of  steepest  ascent  where  the  movement  in  each  design  vaxiable 
direction  is  proportional  to  the  magnitude  of  the  regression  coefiicient  corresponding 
to  that  design  variable  with  the  direction  taken  being  the  sign  of  the  coefiicient 

3.  Conduct  experimental  runs  along  the  path 

4.  Choose  a  new  design  location  where  an  approximation  of  the  maximum  response  is 
located  on  the  path 

5.  Conduct  a  second  fractional  factorial  experiment  centered  at  the  new  design  location 
and  attempt  to  fit  another  first  order  model 

If  a  second  first  order  model  is  accurately  fit,  then  a  second  path  of  steepest  ascent 
can  be  computed  and  traversed  until  a  region  is  reached  where  a  higher-order  model  is 
required  to  accurately  predict  system  behavior. 

In  this  study,  after  the  initial  fractional  factorial  design  was  completed,  a  first  order 
model  was  never  adequate.  Therefore,  the  method  of  steepest  ascent  was  never  required 
because  the  starting  region  of  design  seemed  to  always  be  close  enough  to  the  optimal 
point  over  all  responses. 

Phase  2.  The  purpose  of  this  phase  is  to  build  second  (or  higher)  order 
models  to  accurately  predict  all  responses  and  choose  the  settings  for  the  independent 
variables  that  will  result  in  the  optimal  expected  performance  over  all  responses.  Since  the 
resolution  V  fractional  factorial  was  already  completed  at  the  appropriate  design  point,  the 
ideal  second  order  design  would  be  able  to  simply  augment  the  first  design  to  decrease  the 
total  number  of  rims,  thereby  saving  time  and  money.  Therefore,  the  central  composite 
design  (CCD)  was  used.  A  sample  of  this  design  is  located  in  appendix  B  on  page  B-1. 
This  design  requires  three  parts: 

•  Two-level  factorial  design  or  resolution  V  fraction 
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•  2k  axial  or  star  runs  (k  =  #  of  independent  Vciriables) 

•  Center  runs 

The  resolution  V  fraction  contributes  to  the  estimation  of  the  linear  terms  and  two- 
factor  interactions.  It  is  variance-optimal  for  these  terms.  The  axial  points  contribute  to 
the  estimation  of  the  quadratic  terms.  The  center  rims  provide  an  internal  estimate  of  error 
(pure  error)  and  contribute  toward  the  estimation  of  quadratic  terms.  Since  the  phase  one 
experiments  required  the  fractional  factorieil  design  and  the  center  rims,  to  complete  the 
CCD  the  axial  runs  were  edl  that  was  required. 

The  areas  of  flexibility  in  the  use  of  the  central  composite  design  resides  in  the 
selection  of  a,  the  axial  distance,  and  the  number  of  center  runs.  According  to  Myers  and 
Montgomery  [14]  and  Box  and  Draper  [3],  the  CCD  that  is  most  eflective  from  a  variance 
point  of  view  is  to  use  a  =  Vk  and  three  to  flve  center  runs.  This  design  is  not  necessarily 
rotatable  but  is  near-rotatable.  Therefore,  the  four  center  runs  completed  in  the  initial 
augmented  fractional  factorial  design  were  sufficient  for  the  CCD,  and  the  10  additional 
axial  nms  at  a  =  y/5  =  2.236  were  all  that  were  required  to  complete  the  CCD. 

The  result  of  the  CCD  is  four  models,  one  for  each  of  the  individual  responses. 
Each  model  will  be  a  maximum  of  second  order.  Sample  models  and  their  accompanying 
ANOVA  tables  are  located  in  appendix  B,  starting  on  page  B-2. 

Once  all  models  were  determined,  a  mechanism  for  choosing  the  values  of  the  indepen¬ 
dent  variables  that  would  result  in  the  most-optimal  mission  eff^ectiveness  for  all  responses 
had  to  be  determined.  Because  of  the  complexity  and  miilti-dimensionality  of  the  response 
surfaces,  a  simple  overlaying  of  contour  plots  to  graphiccilly  choose  the  point  which  ap- 
pccired  to  be  optimal  over  all  responses  was  not  applicable.  Therefore,  the  Derringer  and 
Suich  [5]  desirability  function  for  optimizing  over  multiple  responses  was  employed.  This 
method  cillows  for  the  creation  of  desirability  functions  {di,d2,d3,d4)  for  each  response 
where  the  desirability  function  cam  target  a  specific  value,  minimize  or  maximize  a  re¬ 
sponse.  Since  all  the  responses  in  this  study  were  measures  of  mission  effectiveness,  the 
desirability  functions  used  were  all  maximizing.  The  shape  of  each  desirability  function 
is  based  on  the  weight  assigned  to  it  in  accordance  with  Figure  4.1.  Notice  that  as  the 
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weight  increases,  the  desirability  assigned  is  lower  xmtil  the  value  nearly  approaches  the 
goal  (maximum),  and  vice  versa  as  the  weight  decreases. 


Figure  4.1  Desirability  Curves  for  Goal  is  Maximum 

Once  each  individual  desirability  function  is  determined,  a  single  composite  response 
(D)  is  developed  which  is  a  weighted  mean  of  the  desirabilities  of  the  individual  responses. 
The  weights  in  the  composite  response  allow  more  emphasis  on  specific  individual  responses 
as  specified  by  their  importance.  If  all  four  responses  were  of  equal  importance,  D  would 
be  computed  by  equation  (4.2). 


(di-(i2d3<i4)' =  (4.2) 

In  this  study,  extra  emphasis  was  placed  on  two  of  the  responses:  the  number  of  rmique 
kills  and  the  number  of  hits  on  priority  one  targets  (target  formula).  Table  4.3  shows  the 
ratings  using  a  scale  of  one  to  five  of  the  importance  of  ecich  of  the  responses. 

Table  4.3  Importance  Ratings  for  Each  Response 


# 

Response 

Importance 

1 

Unique  Kills 

4 

2 

Total  Kills 

3 

3 

Total  Hits 

3 

4 

Target  Formula 

5 

The  composite  response  (D)  was  then  computed  using  equation  (4.3)  taking  into 
accoimt  the  appropriate  weights  on  importance. 
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D  =  (dr^  •  da"*  •  da"*  -  ^  (  H  j  *  =  (^i'‘ '  ^2^  '  “  (4.3) 


To  determine  the  “optimal”  conditions,  the  following  non-linear  algorithm  with  con¬ 
straints  was  used; 


1.  Let  X  be  a  vector  of  Xj  for  i  =  l...n  representing  the  design  variables  over  the 
optimization  space  which  is  a  subset  of  the  design  space  (in  this  case  n  =  5). 

2.  Let  yj,  Uj,  and  Lj  for  j  =  l...m  be  responses  with  upper  and/or  lower  bounds 
serving  eis  constraints  (in  this  case  m  =  4). 

3.  Let  D{X)  be  the  response  to  be  optimized.  Then  f{X)  =  —D{X)  for  maximization. 

4.  Define  the  constraints  as  a  series  of  discontinuous  functions: 


yj{X)  -  Uj  for  yj  >  Uj 


9j{X)  = 


<  0 


for  Lj  <  yj  <  Uj 


(  Lj  -  yj{X)  for  yj  <  Lj 


This  produces  a  system  of  m  constraints  that  can  be  solved  as  an  unconstrained 
problem  via  a  penalty  function  approach. 

5.  Define  the  cost  function  as: 


{/(X)-»-pS»^W}  (4.4) 

where  p  is  a  penalty  parameter  greater  than  zero  for  j  =  1 . . .  m. 

Finding  an  initial  feasible  region  can  be  diflicult.  The  design  software  used  starts  with  a 
small  value  of  a  penalty  function  in  a  downhill  simplex  (Nelder-Mead)  multi-dimensional 
pattern  search  which  converges  at  either  a  stationciry  point  or  a  design  space  boundary 
[15].  The  search  around  the  initial  convergence  point  is  restarted  using  a  larger  penalty 
function.  Convergence  is  achieved  when  the  distance  moved  or  objective  function  change 
is  less  than  a  10~®  ratio. 
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The  starting  N+1  simplex  points  are  constructed  by  adding  or  subtracting  a  freiction 
of  each  of  the  N  factor  ranges  to  the  initial  starting  point  [15].  The  decision  to  add  or 
subtract  is  made  to  maintain  a  maximum  distance  from  the  factor  limits. 
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V.  Results  and  Analysis 

5.1  Quantitative  Results  and  Analysis 

Specific  numerical  results  are  shown  for  foxur  scenarios  where  a  cooperative  engage¬ 
ment  decision  algorithm  employing  the  optimal  settings  resulted  in  overall  improvement 
over  baseline  (non-cooperative)  performance.  Eaich  scenario  was  defined  by  three  genereil 
characteristics: 

1.  Warhead  lethality 

2.  ATR  precision 

3.  Battlefield  characteristics 

The  specific  parameters  that  were  varied  in  the  simulation  to  define  the  three  general 
characteristics  above  were: 

1.  Probability  of  Kill  (P*) 

2.  False  target  attack  rate  (FTAR)  and  probability  of  target  report  (Pth) 

3.  Clutter  density  {q)  and  whether  the  targets  were  clustered  or  widely  dispersed 

The  battlefield  used  for  all  simulations  was  approximately  300  square  kilometers  in 
size.  Two  groups  of  four  submimitions  each  (totaling  eight  submunitions)  where  employed 
in  all  scenarios.  Each  of  the  groups  flew  a  serpentine  pattern  that  covered  the  entire 
battlefield  in  approximately  20  minutes.  Each  scenario  had  a  total  of  eight  real  targets 
(three  high  priority  emd  five  low  priority).  Also,  two  non- targets  were  employed  in  the 
vicinity  of  the  real  targets  and  a  battlefield  q  of  0.05  per  square  kilometer  were  randomly 
placed  throughout  the  battlefield  in  all  scenarios  (this  value  was  changed  during  the  FTAR 
sensitivity  analysis  described  in  Section  5.2). 

Table  5.1  shows  the  parameters  defining  scenario  1.  This  submunition  has  a  relatively 
non-lethal  Wcurhead  and  is  searching  for  targets  clustered  in  a  four  square  kilometer  region 
of  the  battlefield. 
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Table  5.1  Scenario  1  Defining  Parameters 


Parameter 

Value 

Pit 

0.5 

FTAR 

0.0053  per  square  km 

Pth 

0.95 

Target  Layout 

Cluster 

The  RSM  described  in  section  4.2  was  performed  on  this  scenario  to  determine  the 
ideal  weighting  parameters  for  the  decision  rule  shown  in  equation  (3.2).  When  performing 
the  RSM,  each  simulation  run  required  was  reported  as  a  summary  of  200  Monte  Carlo 
nms.  Each  repetition  was  completed  using  a  different  baseline  seed  for  the  Monte  Carlo 
simulation.  The  resulting  parameters  are  shown  in  Table  5.2.  Note  that  time  of  flight  is 
by  far  the  dominant  term  in  the  optimal  decision  rule  for  this  scenario,  whereas  target 
priority  and  number  of  engaged  munitions  have  little  to  no  influence. 

Table  5.2  Idecil  Parameters  for  Scenario  1  Decision  Algorithm 


Variable 

Weight  on 

Ideal  Value 

ai 

Time  of  Flight 

0.77 

a2 

Target  Priority 

0.14 

Range  Rate 

0.35 

a4 

Number  of  Munitions 

0.0 

Maximum  Commimications  Range 

9.8  km 

The  expected  performance  of  the  wide  area  search  munitions  employing  the  decision 
algorithm  with  the  ideal  weighting  parameters  and  ideal  maximum  commimications  range 
was  then  compared  to  their  baseline  performance  (no  cooperation).  Table  5.3  shows  these 
results  for  each  of  the  responses.  The  overall  percent  improvement  is  simply  an  average 
of  the  percent  improvements  corresponding  to  each  of  the  four  responses.  This  scenario 
showed  significant  room  for  improvement  through  cooperative  behavior. 

Table  5.4  shows  the  parameters  defining  scenario  2.  This  submimition  has  a  lethal 
warhead  and  is  searching  for  targets  clustered  in  a  four  square  kilometer  region  of  the 
battlefield  (same  battlefield  as  scenario  1). 

The  RSM  described  in  section  4.2  was  performed  on  this  scenario  to  determine  the 
ideal  weighting  parameters  for  the  decision  rule  in  a  similar  manner  to  that  for  scenario  1. 
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Table  5.3  Scenario  1  Results 


Response 

No  Cooperation 

Ideal  Cooperation 

Improvement 

Unique  Kills 

2.7 

2.81 

4.07% 

Total  Kills 

3.06 

3.37 

10.13% 

Total  Hits 

6.08 

6.515 

7.15% 

Formula 

8.04 

8.72 

8.46% 

Overall 

7.45% 

Table  5.4  Scenario  2  Defining  Parameters 


Parameter 

Value 

Pit 

0.8 

FTAR 

0.0053  per  square  km 

Ptr 

0.95 

Target  Layout 

Cluster 

The  resulting  parameters  are  shown  in  Table  5.5.  Note  a  more  even  distribution  amongst 
the  first  three  weighting  parameters  with  number  of  engaged  munitions  having  no  influence. 

Table  5.5  Ideal  Parameters  for  Scenario  2  Decision  Algorithm 


Variable 

Weight  on 

Ideal  Value 

ai 

Time  of  Flight 

0.30 

OC2 

Target  Priority 

0.36 

OC3 

Range  Rate 

0.42 

ai 

Number  of  Mxmitions 

0.0 

Maximum  Communications  Range 

20.3  km 

The  same  performance  measurements  as  in  scenario  1  were  analyzed  for  this  scenario. 
Table  5.6  shows  these  results  for  each  of  the  responses.  While  the  performance  gains  were 
not  as  significant  for  this  case,  improvement  of  5-8%  was  still  possible. 

The  only  diflTerence  between  the  first  two  scenarios  was  the  parameter  describing  the 
warhead  lethality  (Pjt).  Therefore,  one  would  expect  the  number  of  hits  on  targets  for 
both  scenarios  to  be  approximately  the  same  (variations  arise  from  stochastics),  and  this 
is  the  case.  However,  the  number  of  kills  would  logically  increase  as  the  lethality  of  the 
warhead  is  increased.  Therefore,  the  number  of  targets  killed  is  greater  in  scenario  2  than 
in  scenario  1,  but  the  incorporation  of  cooperation  engagement  does  not  appear  as  useful 
to  scenario  2.  This  is  also  explained  by  the  difference  in  warhead  lethality.  The  system 
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Table  5.6  Scenario  2  Results 


Response 

No  Cooperation 

Ideal  Cooperation 

Improvement 

Unique  Kills 

4.13 

4.18 

1.21% 

Total  Kills 

4.95 

5.25 

6.06% 

Total  Hits 

6.145 

6.53 

6.27% 

Formula 

8.11 

8.77 

8.11% 

Overall 

5.42% 

with  the  non-lethal  warhead  can  benefit  most  when  additional  strikes  are  mcide  against 
the  same  targets  in  order  to  increase  the  probability  of  killing  them.  However,  in  the  ceise 
where  the  warhead  is  more  lethal,  these  additional  strikes  are  not  as  necessary  and  often 
result  in  wasted  submunitions  attacking  targets  that  cire  already  dead.  An  interesting  point 
is  that  warhead  lethality  is  often  “traded  off”  with  size  and  cost  in  order  to  achieve  smaller 
and  cheaper  munitions.  Employing  smaller  munitions  provides  the  aircraft  the  capability 
to  carry  more  mimitions.  The  results  from  these  scenarios  demonstrate  that  cooperative 
engagement  may  be  able  to  compensate  for  some  of  the  lethality  lost  by  choosing  smaller, 
cheaper  munitions. 

Table  5.7  shows  the  parameters  defining  scenario  3.  This  submunition  has  a  relatively 
non-lethal  warhead  and  is  searching  for  targets  widely  dispersed  throughout  the  entire 
battlefield. 


Table  5.7  Scenario  3  Defining  Parameters 


Parameter 

Value 

Pfc 

0.5 

FTAR 

0.0053  per  square  km 

Ptr 

0.95 

Target  Layout 

Widely  Dispersed 

The  same  RSM  as  the  previous  scenarios  was  performed  on  this  scenario.  The  result¬ 
ing  parameters  are  shown  in  Table  5.8.  Time  of  flight  is  still  the  most  important  parameter, 
just  as  in  scenario  1,  but  for  this  scenario  target  priority  becomes  more  important  than 
range  rate. 

Table  5.9  shows  the  results  for  each  of  the  responses. 
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Table  5.8  Ideal  Parameters  for  Scenario  3  Decision  Algorithm 


Variable 

Weight  on 

Ideal  Value 

Oil 

Time  of  Flight 

0.71 

OC2 

Target  Priority 

0.48 

Range  Rate 

0.1 

Oi4 

Number  of  Mimitions 

0.1 

Maximum  Commimications  Rcinge 

13.3  km 

Table  5.9  Scenario  3  Results 


Response 

No  Cooperation 

Ideal  Cooperation 

Improvement 

Unique  Kills 

2.72 

2.70 

-0.74% 

Total  Kills 

3.07 

3.35 

9.12% 

Total  Hits 

6.295 

6.52 

3.57% 

Formula 

8.56 

9.245 

8.00% 

Overall 

4.99% 

In  this  scenario  where  the  targets  were  widely  dispersed,  cooperative  engagement 
appears  to  have  less  utility  than  when  the  targets  were  clustered  (scenarios  1  and  2).  This 
is  mostly  due  to  the  number  of  unique  kills.  The  cooperative  engagements  are  good  for 
putting  additional  hits  on  known  targets,  but  they  limit  the  total  search  area  covered  by 
the  submunitions.  When  the  targets  are  widely  dispersed,  the  unique  kills  are  increased 
not  only  by  putting  additional  hits  on  known  targets,  but  also  by  seaxching  the  entire 
battlefield.  It  appears  that  the  increase  in  hits  on  targets  is  slightly  less  beneficial  (in 
terms  of  unique  kills)  than  continuing  to  search  the  entire  battlefield  for  more  targets 
when  the  targets  are  widely  dispersed.  When  the  targets  axe  clustered,  a  submimition 
that  cooperatively  engages  a  known  target  area  has  a  pretty  good  chance  of  encountering 
a  unique  target  in  the  vicinity  of  the  known  target.  This  results  in  more  unique  hits,  but 
this  is  not  apparent  in  any  of  the  four  responses  evaluated  in  this  study.  Therefore,  the 
total  number  of  kills  and  hits  are  pretty  similar  for  the  clustered  target  scenarios  and  the 
widely  dispersed  target  scenarios. 

Table  5.10  shows  the  parameters  defining  scenario  4.  This  submimition  has  a  lethal 
warhead  cind  is  searching  for  targets  that  are  widely  dispersed  throughout  the  entire  bat¬ 
tlefield  (same  battlefield  as  scenario  3). 
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Table  5.10  Scenario  4  Defining  Parameters 


Parameter 

Value 

Pjfe 

0.8 

FTAR 

0.0053  per  square  km 

Ptr 

0.95 

Target  Layout 

Widely  Dispersed 

The  resulting  parameters  after  completing  the  RSM  axe  shown  in  Table  5.11.  Note 
that  this  scenario  has  the  most  evenly  distributed  weighting  parameters  of  all  the  scenarios 
considered. 


Table  5.11  Ideal  Parameters  for  Scenario  4  Decision  Algorithm 


Variable 

Weight  on 

Ideal  Value 

ai 

Time  of  Flight 

0.31 

Oi2 

Target  Priority 

0.35 

0:3 

Range  Rate 

0.40 

0:4 

Number  of  Munitions 

0.15 

Maximum  Commimications  Range 

19.7  km 

Table  5.12  shows  the  results  for  each  of  the  responses. 


Table  5.12  Scenario  4  Results 


Response 

No  Cooperation 

Ideal  Cooperation 

Improvement 

Unique  Kills 

3.93 

3.99 

1.53% 

Total  Kills 

4.97 

5.3 

6.64% 

Total  Hits 

6.225 

6.555 

5.30% 

Formula 

8.38 

9.03 

7.76% 

Overcill 

5.31% 

The  results  from  scenario  4  corroborate  the  results  from  the  previous  scenarios  very 
well.  The  increase  in  the  warhead  lethality  over  that  for  scenario  3  results  in  some  im¬ 
provement  in  the  number  of  unique  kills,  even  when  cooperative  behavior  is  employed. 
When  the  submimitions  cooperatively  engaged  in  scenario  3,  the  additional  hits  often  did 
not  result  in  a  dead  target-therefore,  not  only  did  the  number  of  kills  not  increase,  but 
less  of  the  battlefield  was  searched.  In  scenario  4,  more  of  these  cooperative  engagements 
resulted  in  dead  targets,  therefore  making  the  tradeoff  of  searching  less  area  a  little  more 
enticing.  However,  for  the  Scime  reasons  as  stated  earlier  on  page  5-3,  the  increase  in 
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hits  on  known  targets  when  the  submunition’s  warhead  is  more  lethal  often  results  in  more 
wasted  submunitions.  This  accounts  for  a  lower  utility  in  using  cooperative  engagement  for 
scenario  4  than  scenario  3,  but  apparently  the  benefits  described  earlier  slightly  outweigh 
this  detriment. 


5,2  Sensitivity  Analysis 

As  the  precision  of  the  ATR  is  degraded  and/or  the  clutter  density  increases,  this  form 
of  cooperative  engagement  does  not  offer  any  advantages  and  often  deteriorates  the  overall 
performance  of  the  wide  area  search  munitions.  This  is  because  of  the  hyper-sensitivity 
to  the  false  target  attack  rate.  By  degrading  the  ATR  precision  and/or  increasing  the 
clutter  density,  FTAR  increases.  Therefore,  what  often  occurs  is  that  a  submunition  falsely 
identifies  a  clutter  or  non- target  as  a  real  target  and  then  communicates  to  some  of  the  other 
munitions  the  existence  of  a  rea/-target  that  doesn’t  actually  exist.  Then  one  or  more  of 
the  other  submunitions  will  decide  to  cooperatively  engage  that  false  target.  Now  the  best 
event  that  could  occur  for  that  redirected  submimition  is  that  it  just  happens  to  encounter 
a  real  target  on  its  fiight  path  to  the  false  target  (the  chances  of  that  event  occurring  being 
no  better  than  if  the  submunition  would  have  just  stayed  on  its  original  search  pattern). 
However,  if  that  does  not  happen,  the  submunition  is  guaranteed  to  encounter  that  false 
target  that  it  thinks  is  a  real  target.  Upon  encountering  the  false  target,  the  submimition 
may  correctly  identify  it  and  not  engage  it,  but  as  FTAR  increases,  this  is  less  and  less 
likely.  Therefore,  cooperative  engagement  alone  cannot  overcome  the  hyper-sensitivity  in 
wide  area  search  munition  effectiveness  to  increasing  FTAR. 

For  a  given  scenario,  if  the  weights  in  the  decision  algorithm  are  chosen  wisely, 
degraded  performance  due  to  cooperative  engagement  can  be  minimized.  To  demonstrate 
this,  a  sensitivity  analysis  was  performed  to  investigate  the  mission  effectiveness  versus  a 
varying  FTAR.  Scenario  1  was  chosen  for  this  analysis  because  cooperative  engagement 
seemed  to  be  most  beneficial  to  this  scenario.  The  simulation  was  run  for  varying  FTAR’s 
and  the  two  most  critical  responses  were  measured  and  analyzed:  unique  kills  and  target 
formula  (equation  (4.1)).  The  weights  in  the  decision  algorithm  (a^’s)  remained  constant 
for  all  runs  and  were  assigned  the  optimal  values  for  scenario  1  shown  in  Table  5.2.  For 
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this  FTAR  sensitivity  analysis,  the  Ptr  remained  constant  at  0.95  and  the  was  either 
0.5  (non-lethal  warhead)  or  0.8  (lethal  warhead).  The  results  for  the  unique  kills  are  shown 
in  Figiure  5.1,  aind  the  results  for  the  target  formula  are  shown  in  Figure  5.2.  The  curves 
representing  varying  P*  are  not  in  Figure  5.2  because  warhead  lethality  does  not  affect 
whether  or  not  the  mTUiition  hits  the  target. 


0.0001  0.001  0.01  0.1  1 

FTAR  (/km*2) 

Figure  5.1  Unique  Kills  Sensitivity  to  FTAR  for  2  Warheads 


FTAR  (/km*2) 


Figure  5.2  Target  Formula  Sensitivity  to  FTAR 


In  both  Figures  5.1  aind  5.2,  cooperative  engagement  very  rarely  deteriorates  the 
performance  of  the  submunitions  when  compared  to  the  baseline  (no  cooperation).  This 
is  because  of  the  choice  of  weight  in  the  decision  algorithm.  Notice  that  the  weight  on 
the  time  of  flight  parameter  in  Table  5.2  is  relatively  high  (0.77).  Because  of  the  relative 
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importance  on  this  parameter  versus  the  others,  the  submunitions  are  basing  the  majority 
of  their  cooperative  engagement  decision  on  time.  Scenario  1  is  defined  by  all  targets  clus¬ 
tered  in  a  small  area  in  the  center  of  the  battlefield.  Therefore,  as  FTAR  gets  higher  and 
higher  (worse  and  worse),  the  submunitions  aren’t  lasting  long  enough  into  the  total  search 
time  to  even  participate  in  cooperative  engagements  because  they  are  falling  for  clutter 
targets.  Therefore,  for  the  cases  of  high  FTAR,  cooperative  engagements  are  very  infre¬ 
quent  resulting  in  similar  outcomes  for  the  cooperative  behavior  and  baseline  situations. 
For  this  case,  if  the  time  of  flight  weight  in  the  decision  rule  is  dropped  in  importance  and 
any  of  the  other  weights  are  increased,  more  cooperative  engagements  occur  early  in  the 
search  patterns  resulting  in  deteriorated  performance. 

What  happens  when  FTAR  remains  low,  but  Ptk  decreases?  This  means  that  given 
real  target  encounters,  the  probability  that  the  ATR  is  correctly  identifying  the  real  targets 
is  decreasing,  i.e.,  there  is  an  increase  in  submunitions  not  engaging  real  targets  because 
they  are  falsely  identifying  them  as  non-targets.  This  situation  is  realistic  because  often 
in  ATR  algorithm  precision  tuning,  F^r  Is  traded  off  with  FTAR.  In  this  situation,  as 
long  as  FTAR  remains  low  (favorable),  cooperation  can  still  improve  overall  effectiveness. 
This  is  because  a  submunition  may  later  encounter  and  correctly  identify  (and  therefore 
communicate  and  engage)  a  real  target  that  another  submunition  may  have  previously 
incorrectly  identified  as  a  false  target.  Then  through  cooperation,  the  submunition  that 
originally  made  the  incorrect  identification  could  go  back  and  get  a  second  look  at  that 
target  and  possibly  correctly  identify  and  engage  it.  A  scenario  such  as  this  will  also  benefit 
from  redundant  area  coverage  with  the  initial  search  patterns  at  the  expense  of  reduced 
total  area  coverage  rate. 

A  sensitivity  analysis  to  Ftr  was  completed  in  a  similar  manner  to  that  for  FTAR. 
This  analysis  was  completed  on  scenario  1  using  the  ideal  decision  rule  weighting  param¬ 
eters  shown  in  Table  5.2.  For  all  runs,  FTAR  remained  constant  at  a  very  low  value  of 
0.007  /km^.  The  same  two  responses  were  looked  at.  The  results  for  the  unique  kills  are 
shown  in  Figure  5.3,  and  the  results  for  the  target  formula  are  shown  in  Figure  5.4.  The 
curves  representing  varying  are  not  in  Figure  5.4  because  warhead  lethality  does  not 
affect  whether  or  not  the  mimition  hits  the  target. 
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ProbTR 

Figure  5.3  Unique  Kills  Sensitivity  to  Ptr  for  2  Warheads 


1  0.75  0.5  0.25  0 

Prob  TR 


Figure  5.4  Target  Formiila  Sensitivity  to  Ptr 
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The  sensitivity  analyses  described  here  could  be  considered  robiistness  analyses  of  the 
weights  in  the  decision  algorithm  to  varying  ATR  algorithms.  What  might  be  of  interest 
for  future  work  is  to  repeat  the  previous  sensitivity  analyses  but  not  keep  the  decision 
jilgorithm  the  same  throughout.  Instead,  for  each  ATR  algorithm  analyzed,  determine  the 
ideal  weighting  parameters  for  the  decision  rule  using  the  RSM  techniques  described  in 
Section  4.2.  Doing  this  should  provide  an  increase  in  the  performance  of  the  cooperative 
systems  over  the  baseline  and  may  provide  some  additional  insight  into  these  sensitivities. 

A  final  analysis  was  completed  to  see  the  sensitivity  to  target  density.  For  this 
aneJysis,  a  scenario  very  similar  to  that  of  scenario  3  was  used  with  a  few  variations. 
Table  5.13  shows  the  specific  characteristics  of  this  scenario. 

Table  5.13  Target  Density  Sensitivity  Scenario  Defining  Parameters 


Parameter 

Value 

Pfc 

0.5 

FTAR 

0.009  per  square  km 

Ptr 

0.80 

Target  Layout 

Widely  Dispersed 

To  do  this  analysis,  the  tzirget  density  was  increased  by  simply  increasing  the  number 
of  targets  in  the  simulation.  A  total  of  nine  sets  of  runs  were  examined  using  the  specific 
target  distribution  described  in  Table  5.14.  The  number  of  high  priority  targets  was  set 
equal  to  the  set  number.  The  number  of  low  priority  targets  was  equal  to  two  times  the 
number  of  high  priority  targets  minus  one.  The  number  of  non-targets  weis  equal  to  one 
less  than  the  munber  of  high  priority  targets. 


Table  5.14  Target  Distribution  for  Each  Set  of  Runs 


Set# 

High  Priority 

Low  Priority 

Total  Real  Targets 

Non-  Target 

1 

1 

1 

2 

0 

2 

2 

3 

5 

1 

3 

3 

5 

8 

2 

4 

4 

7 

11 

3 

5 

5 

9 

14 

4 

6 

6 

11 

17 

5 

7 

7 

13 

20 

6 

8 

8 

15 

23 

7 

9 

9 

17 

26 

8 
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Each  set  of  nuis  consisted  of  200  Monte  Carlo  runs  with  eight  submunitions,  and 
the  number  of  unique  kills  and  the  target  formula  from  equation  4.1  were  measured  for 
both  cooperative  behavior  and  non-cooperative  behavior  (baseline).  Since  the  battlefield 
characteristics  for  this  scenario  were  similar  to  that  of  scenario  3,  the  weights  used  in 
the  cooperative  engagement  decision  rule  were  those  determined  to  be  ideal  for  scenario 
3  (listed  in  Table  5.8).  The  results  for  the  imique  kill  sensitivity  are  shown  in  Figure  5.5, 
and  the  results  for  the  target  formula  sensitivity  are  shown  in  Figure  5.6. 


8  submunitions 


0  5  10  15  20  25  30 

Number  of  Real  Targets 


Figure  5.5  Unique  Kills  Sensitivity  to  Target  Density 

8  submunitions 


0  5  10  15  20  25  30 


Number  of  Real  Taraets 

Figure  5.6  Target  Formula  Sensitivity  to  Target  Density 

Looking  bzick  at  the  final  results  from  scenario  3,  it  is  interesting  to  note  that  the  num¬ 
ber  of  imique  kills  with  cooperation  was  less  than  the  baseline  performance  (see  Table  5.9). 
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Therefore,  it  is  not  too  surprising  to  see  that  in  Figure  5.5  the  results  from  incorporating 
cooperation  remain  slightly  worse  than  the  baseline  performance  results  across  all  target 
densities.  However,  Figure  5.6  shows  that  the  target  formula  (which  emphasizes  hits  on 
high  priority  targets)  is  always  greater  for  the  cooperating  case  than  the  baseline  case.  It 
is  especially  interesting  to  note  that  the  greatest  improvement  seen  by  the  incorporation  of 
cooperative  behavior  is  when  the  number  of  real  targets  in  the  battlefield  is  approximately 
75%  of  the  number  of  submunitions  employed. 

A  different  way  of  doing  the  previous  sensitivity  analysis  would  be  to  vary  the  number 
of  munitions  in  the  simulation  as  well  as  the  number  of  targets.  Additional  munitions  with 
more  overlapping  search  patterns  should  provide  improved  performance,  but  how  much  is 
unknown.  This  information  would  be  very  useful  for  better  understanding  the  relationships 
between  mission  effectiveness,  target  densities  and  target-to-munition  ratios. 


5.3  Robustness 

To  test  the  robustness  of  the  optimal  decision  parameters  determined  for  each  sce¬ 
nario,  the  optimal  decision  rule  for  one  scenario  was  r\m  on  a  different  scenario  and  then 
compared  to  the  baseline  performance.  For  example,  the  optimal  decision  parameters 
for  scenario  1  (as  defined  in  Table  5.2)  were  implemented  in  the  simulation  setup  to  run 
scenario  2  (as  defined  by  the  parameters  listed  in  Table  5.4).  This  was  done  for  all  com¬ 
binations  of  the  four  scenarios  described  in  the  quantitative  residts  section  (section  5.1). 
In  general,  the  results  proved  very  little  robustness  to  the  selection  of  the  optimaJ  deci¬ 
sion  parameters.  In  most  cases,  the  performance  with  the  sub-optimal  decision  peirameters 
resulted  in  a  zero  to  two  percent  overall  improvement  over  baseline  performance,  but  some¬ 
times  resulted  in  deteriorated  performance  when  compared  to  the  baseline. 

With  these  results  an  attempt  was  then  made  to  correlate  the  values  of  the  optimal 
weighting  parameters  to  the  parameters  used  to  define  the  different  scenarios.  The  results 
showed  some  correlation  of  the  general  distribution  of  weights  in  the  decision  algorithms 
to  whether  the  targets  were  clustered  or  widely  dispersed,  but,  due  to  the  diversity  in 
the  optimal  weighting  parameters  across  all  four  scenarios,  no  specific  correlations  were 
recognized.  The  only  parcimeter  that  displayed  some  sort  of  consistency  was  that  associated 
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with  the  fuel  remaining  or  time  of  flight— there  appears  to  be  some  value  in  waiting  until  the 
latter  part  of  the  sejurch  pattern  to  choose  to  cooperatively  engage  a  known  target.  This, 
of  comrse,  makes  sense  and  allows  for  the  greatest  exploration  of  the  entire  battlefield. 
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VI.  Conclusions  and  Recommendations 

This  research  established  a  methodology  for  measuring  the  expected  effectiveness  of  a 
cooperative  system  of  wide  area  search  mimitions.  The  methods  used  in  this  research 
are  not  limited  to  any  particular  type  of  wide  area  search  munition  and  were  consciously 
completed  using  parameters  that  describe  a  very  generic  wide  area  search  munition.  This 
research,  therefore,  applies  to  all  wide  area  search  munitions  and  other  cooperative  vehicles, 
and  more  specific  results  can  be  achieved  for  any  specific  system  by  simply  modifying  the 
parameters  in  the  effectiveness  simulation.  Further,  the  methods  developed  as  part  of  this 
research  have  applications  in  the  more  general  area  of  cooperative  behavior  and  control. 

The  form  of  cooperative  engagement  used  in  this  study  is  most  useful  in  overcoming 
the  limitations  on  warhead  lethality.  As  the  munition  size  and  cost  are  decreased,  the 
warhead  is  likely  to  become  less  and  less  lethal.  This  research  shows  that  some  of  this  loss 
in  lethality  can  be  made  up  by  the  use  of  cooperative  engagement.  Also,  as  submimition 
size  decreases,  the  number  of  weapons  carried  by  an  aircraft  increases.  This  research 
hints  at  the  possibility  of  achieving  greater  results  with  higher  numbers  of  cooperative 
munitions  carrying  non-lethal  warheads  rather  than  fewer  submimitions  with  larger,  more 
lethal  warheads. 

Cooperative  engagement  employed  in  this  study  demonstrated  the  potential  ability  to 
overcome  lower  Ptr’s.  However,  cooperative  engagement  alone  is  not  able  to  compensate 
for  higher  false  target  attack  rates.  When  tuning  the  precision  of  an  ATR  algorithm,  the 
FTAR  is  often  traded-off  with  Ptr-  As  the  algorithm  is  opened  up  allowing  less  of  a  match 
to  predetermined  target  images  for  target  identification  purposes,  then  Ptr  is  increased 
but  the  munber  of  false  alarms  is  also  increased.  On  the  other  hand,  when  the  algorithm 
is  tuned  so  tight  requiring  almost  perfect  matches,  the  FTAR  is  decreased  but  some  real 
targets  are  missed  {Ptr  is  decreased).  This  research  hints  that  when  the  ATR  is  being 
tuned,  low  FTAR  should  be  favored  8ind  the  decrease  in  can  possibly  be  overcome 
through  cooperative  engagement.  Also,  the  selection  of  the  optimal  weights  in  the  decision 
algorithm  are  very  sensitive  to  all  battlefield  characteristics. 
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Sensitivity  analyses  show  that  for  most  scenarios,  cooperative  engagement  can  pro¬ 
vide  improvement  over  completely  autonomous  performance  when  the  weighting  parame¬ 
ters  in  the  decision  algorithm  are  chosen  wisely.  These  analyses  specifically  identify  regions 
of  operation  where  this  form  of  cooperative  engagement  is  most  effective. 

To  improve  the  results  of  this  research,  additional  studies  on  cooperative  search  and 
cooperative  discrimination  must  be  included  with  the  cooperative  engagement  algorithm 
to  better  achieve  the  full  synergistic  value  of  cooperative  wide  area  search  munitions.  Addi¬ 
tionally,  various  decision  algorithms  should  be  explored.  For  example,  different  paxameters 
could  be  included  in  the  decision  algorithm  such  as  range  from  target  to  submxmition,  and 
different  decision  rule  forms  (second  order,  inclusion  of  interaction  terms,  etc.)  can  be 
explored  to  improve  the  effectiveness  of  cooperative  engagement.  Further,  the  methods  by 
which  each  of  the  specific  parameters  in  the  simulation  were  normalized  can  be  evaluated 
to  possibly  reduce  the  sensitivities  of  the  ideal  weights  in  the  decision  algorithms.  How¬ 
ever,  even  without  any  of  these  modifications,  some  scenarios  most  likely  do  exist  where 
the  form  of  cooperative  engagement  employed  by  this  research  will  provide  more  signifi¬ 
cant  improvement  than  actually  demonstrated  by  the  results  of  this  study.  Although  little 
investigation  has  been  completed,  this  research  hints  at  improved  performance  in  a  more 
target-rich  scenario  with  multiple  groups  of  clustered  targets  where  the  warhead  lethality 
of  the  mimitions  is  low.  More  investigations  into  diverse  scenarios  could  possibly  identify 
cases  where  this  form  of  cooperative  engagement  can  be  more  useful;  and  equally  impor¬ 
tant,  more  scenarios  can  be  identified  where  this  form  of  cooperative  engagement  is  not 
applicable  and  causes  a  deterioration  in  overall  mission  effectiveness. 

An  interesting  modification  to  the  simulation  could  be  to  allow  a  submunition  to 
find  a  target  and  communicate  it,  but  not  engage  it.  As  the  simulation  currently  runs, 
once  a  target  is  identified,  the  munitions  automatically  engages  it.  Altering  this  behavior 
could  allow  multiple  munitions  to  take  additional  “looks”  at  targets  to  better  identify  them 
before  engaging  them  and  can  allow  individual  submunitions  to  explore  larger  portions  of 
the  battlefield  prior  to  making  an  engagement. 

Additional  methods  of  optimization  could  be  explored  for  choosing  the  ideal  weights 
in  the  decision  algorithm.  For  example,  the  parameters  resulting  in  a  maximum  for  one  of 
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the  responses  (found  by  simple  stationary  point  analysis)  Ccin  be  compared  to  those  foimd 
by  the  Derringer  and  Suich  desirability  function  method.  Also,  variations  in  the  choice 
of  weights  for  each  response  in  the  optimization  routine  used  in  this  research  could  be 
explored  to  possibly  find  “more  optimal”  solutions. 

Further  sensitivity  ancJyses  could  be  conducted  using  the  ideal  weights  in  the  decision 
rule  for  each  variation  in  the  precision  of  the  ATR  algorithm  instead  of  keeping  the  weights 
constant.  Additionally,  the  sensitivity  analysis  for  the  target  density  could  be  improved  by 
varying  the  number  of  submtmitions  in  addition  to  varying  the  number  of  targets.  This  can 
also  be  modified  using  various  distributions  of  high  and  low  priority  targets  and  non-targets 
as  well  as  varying  the  number  of  mobile  versus  stationary  targets.  Finally,  more  sensitivity 
information  could  be  garnered  using  various  RSM  techniques  to  better  understand  how  to 
choose  the  weights  in  the  decision  algorithm  based  on  the  scenario  characteristics  without 
having  to  go  through  an  entire  DOE  and  RSM  study. 
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Appendix  A.  Sample  PSub  Files 

A,1  Input  Files 

A. 1.1  Main  Input  File. 


$RUN_DATA 

RUN.COMMENTS 

DIS 

BLIND.IM.TURNS 
no. sub. types 
no.pattern. types 
NO.REPS 
XSEED 

SIM.TIME.STEP 

OUTPUT.TGT.STATS 

OUTPUT.HIST 

OUTPUT.CUTS 

OUTPUT.PLAYBACK 

PRNT.INTRVL 

NO.ROUNDS 

R0UND.REL 

USE.CEP 

TLE(l) 

DISP.CEP(l) 

DISP.PREC(l) 

AIHX(l) 

AIMY(l) 

paos 

RND.TIME.DISP(l) 

TGT.AREA.DEF 

XBFHAX 

XBFMIN 

YBFMAX 

YBFMIN 

RANDOM.TARGETS 

TARGET.XMAX 

TARGET.XMIN 

TARGET.YMAX 

TARGET.YMIN 

NO.TARGETS 

$END 


=  »D0E  Test  #1» 

=  .F.,  !  RUNNING  Distributed  Interactive  Simulation 

=  .T.,  !  switch  to  turn  on/off  seeker  in  turns 

=  1,  !  if  dis  =  .t.,  #  seeker.inp  sections  in  this  file 

~  4,  !  if  dis  =  .t,,  #  patt.inp  sections  in  this  file 

==  200,  !  NUMBER  OF  MONTE  CARLO  TRIALS 

=  » 012342’  !  MONTE  CARLO  SEED  (TRIAL  #  WILL  BE  APPENDED)  JCS  MODIFIED  FROM  SEED 
=1.0,  !  SIMULATION  TIME  STEP 
=  .T.,  !  OUTPUT  ACQ, HIT, KILL  DATA  FOR  TARGET 

=  1,  !  TRIAL  #  for  OUTPUT  ENTIRE  BATTLE  HISTORY 

=  1,  !  TRIAL  #  for  OUTPUT  ENTIRE  ENTRY  AND  EXIT  TIMING 

=  0,  !  TRIAL  #  for  OUTPUT  PLAYBACK  TRAJECTORIES. 

=  1000,  !  MONITOR  MONTE  CARLO  MOD  NUMBER 

=  1,  !  NUMBER  OF  ROUNDS  OR  DISPENSERS 

=1.0,  !  ROUND  RELIABILITY 

=  .T.,  \  IF  .T.,  1ST  VALUES  IN  CEP  .F.  -  BOTH  VALUES 

=100.0,100.0,  !  TLE  RANGE  AND  DEFLECTION 

=13.0,0.0,  !  CEP  OR  MPI  BIAS  RANGE  AND  DEFLECTION 

=  1.0, 1.0,  !  PREC.  RANGE  AND  DEFLECTION 

=0.0,  !  DISPENSERCOR  ROUND)  AIM  X 

=0.0,  !  DISPENSERCOR  ROUND)  AIM  Y 

=  1.00 

=  0,  !  DISPENSERCOR  ROUND) TIME  FROM  DISPENSE  TO  BEGIN  SEARCH 

=  225000,  !  TGT  AREA  DEFINITION : 0-DISREGARD ,POS-USE  AREA , NEG-CHECK  TGTS 

=  15000.0,  !  BATTLEFIELD  DIMENSIONS 

=-2000.0, 

=  15000.0, 

=-2500.0, 

=  .T.,  !  PLACE  TARGETS  RANDOMLY  WITHIN  A  TARGET  AREA  INPUT  BELOW 

=  5000.0, 

=  7000.0, 

=  5000.0, 

=  7000.0, 

=  10, 


$SEEKER.INP  !  SUBMUNITION  TYPE  /  SEEKER  DATA  INPUT... 

ROUNDS (1)  =-l.  !  ROUNDS  THAT  USE  THIS  SUB/SEEK  DATA  -1  ALL 
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SUB.TYPE  =  'LC’  !  SUBMUNITION  TYPE  NAME 

SEEKER_TYPE  =  'broomsweep* ,  !  STARE;  BROOMSUEEP;  CURVESWEEP 

SUB_REL  =0.95,  !  SUBMUNITIQN  RELIABILITY 

FTP.WIDTH  =  500. ,  !  FOOT  PRINT  WIDTH 

SEEK_BMV  =3.4,  !  BEAM  WIDTH  (DEG.) 

SEEK_DEPRSS  =13.0,  !  BORESIGHT  ANGLES  HEAS.  +DQWN  OFF  HORIZ. 

SEEK_SCAN_TIME  =  1.8,  !  TIME  fov  TO  SWEEP  FTP.WIDTH 

SEEK_FLYBACK_TIME  =0.2, 

OVERLAP.DESIREDd)  =  0.  ,100.  ,200. ,  !  OVERLAP  DIST.  FOR  SWEEP  DESIRED  (D  this  vel  ft  ALT 

MIM_TIME_ACQ  =0.001,  !  MINIMUM  TIME  IN  FOV  TO  ACQUIRE  A  TARGET 

MIN_TURN_RADIUS  =  270. ,  !  MINIMUM  TURN  RADIUS  OF  SUBMUNITION. 

COOP  =  1,  !  Added  By  Dan  Gillen,  15  Sep  00,  for  unique  cooperative  rule 

RANGEPAR  =  0.1,!  Added  By  Dan  Gillen,  Range  Parameter  for  decision  rule 
TOFPAR  =  0.71,!  Added  By  Dan  Gillen,  Time  Parameter  for  decision  rule 
PRIORPAR  =  0.48, !  Added  By  Dan  Gillen,  Priority  Parameter  for  decision  rule 
N0_ENG_PAR  =  0.1,!  Added  By  Dan  Gillen,  #  Engaged  Parameter  for  decision  rule 
C0MN_RANGE  =  9806.6,!  Added  By  Dan  Gillen 

COOPERATIVE (1)  =0,  !  0  -  NO  COOP;  1-GO  NOW;  2-GO  ffl  END  OF  FP 

COOPERATIVE (2)  =0,  !  #  SUBS  CALL:  0  -  1/TGT  SEEN;  1-ALL;  2-HALF ; 3-THIRD. . . 

COOPERATIVE (3)  =0,  !  1  -  «  END  OF  FP,SEND  REMAINING  SUBS  TO  FIRST  TARGET  AREA;  0  -  DON’T 

SELECT_C0UNTER(1)  =1,1, 

SELECT. AFT.HDNG  =  0,  !  HDNG  CHANGE  THAT  TGT  SELECTION  SHOULD  BEGIN  AFTER 

CORRELATE.TGTS  =  .F.,  !  CAN  SUBMUNITION  TELL  IF  IT'S  SEEN  A  TGT  BEFORE 

NTGT.TYPES  =  4, 

TGT.TYPEd)  =  ’PRIIE’,  ’PRI2L’ , 'PRI3A’ .  ’CLUTTER’, 

TGT_PLIVEd)  =  0.0, 0.0, 0.0, 0.0, 

TGT_PRIORd)  =  1,2,99,99, 

TGT_PAd)  =  1.0, 1.0, 1.0, 1.0, 

TGT_PHd)  =  1.0, 1.0, 1.0, 1.0, 

TGT_PKd)  =  0.5, 0.5, 0.5, 0.5, 

TGT_PIDd,l)  =  0.95,0.016666,0.016666,0.016666, 

TGT_PIDd,2)  =  0.0125,0.95,0.025,0.0125, 

TGT_PIDd,3)  =  0.0125,0.025,0.95,0.0125, 

TGT_PID(1,4)  =  0.0475,0.0475,0.0,0.905, 

CLUTTER.DENSITY  =0.05,  !  CLUTTER  TARGET  DENS.  (PER  KM**2) 

$END 

IPATT.INP  !  PATTERN  DATA  INPUT. . . 

ROUNDSd)  =  0,  !  ROUNDS  THAT  USE  THIS  PATTERN  DATA  -1  ALL 

TYPE  =  ’RACETRAK’,  !  SPIRAL;  CIRCLE;  LINE;  RACETRAK;  DUMB 

RED_TGT_HDNG  =  .F., 

POWERED  =  .T.,  !  ARE  SUBS  POWERED  (  .false.  =  glide  or  dumb) 

RACETRACK.LENGTH  =  4500., 

RACETRACK.WIDTH  =  4000., 

RACETRACK_NO_REVS  =  2, 

FTP.TOF  =  1680.,  !  or  END  OF  FOOT  PRINT  CUT  OFF;  TOF 
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SUB^ALT 

SUB.VEL 

NO^SUBS.IN.PAT 

DISP.OPTION 

DISP.RNGE 

DISP^DEFL 

DISP_HEADING 

DISP.DELAY 

$EtND 

$PATT_INP 


$END 

$PATT.INP 


$ENB 

SPATT.INP 
ROUNDS (1) 
TYPE 

RED^TGT.HDNG 

POWERED 

FTP.TOF 

SUB.ALT 

SUB.VEL 

CENTER! 

CENTERY 


300.  , 

100., 

4. 

'U.RLINE*, 

1500., 

0., 

0., 

0.0, 


!  SUBMUNITION  ALTITUDE 
1  METERS/SEC. 

!  #  SUBMUNIIONS  IN  PATTERN/ROUND 

!  BIVNORML;  R^RLINE;  U.RLINE ; R^DLINE ;  U.DLINE;  NONE 


!  DISPENSE  HEADING  (SUB’S  INITIAL  HDNG) 
!  DELAY  BETWEEN  SUBS  OUT  OF  DISPENSER 


ROUNDS (1) 

=  0, 

!  ROUNDS  THAT  USE  THIS  PATTERN  DATA  -1  ALL 

TYPE 

=  ’DUMB’, 

!  SPIRAL;  CIRCLE;  LINE;  RACETRAK;  DUMB 

RED.TGT_HDNG 

=  .F., 

POWERED 

=  .F., 

!  ARE  SUBS  POWERED  (  .false.  =  glide  or  dumb) 

LETHAL^RADIUS 

«  25.0, 

!  for  dumb  munitions 

ND_SUBS^IN_PAT 

=  380, 

!  #  SUBMUNIIONS  IN  PATTERN/ROUND 

DISP.OPTION 

=  ’bivnorml’ , 

!  BIVNORML;  R.RLINE;  U_RLINE;R_DLINE;  U.DLINE; 

DISP.RNGE 

=  200. . 

DISP.DEFL 

=  200., 

DISP.DELAY 

=  0.0, 

!  DELAY  BETWEEN  SUBS  OUT  OF  DISPENSER 

ROUNDS (1) 

=  -1, 

!  ROUNDS  THAT  USE  THIS  PATTERN  DATA  -1  ALL 

TYPE 

=  ’LINE’, 

!  SPIRAL;  CIRCLE;  LINE;  RACETRAK;  DUMB 

POWERED 

=  .T., 

!  ARE  SUBS  POWERED  (  .false.  =  glide  or  dumb) 

FTP_TOF 

=  1200,  ! 

or  END  OF  FOOT  PRINT  CUT  OFF;  TOF 

SUB.ALT 

=  200. , 

!  SUBMUNITION  ALTITUDE 

SUB.VEL 

=  100., 

!  METERS/SEC. 

NO.SUBS.IN.PAT 

=  8, 

!  #  SUBMUNIIONS  IN  PATTERN/ROUND 

DISP.OPTION 

=  ’U.DLINE’; 

,  !  BIVNORML;  R.RLINE;  U_RLINE;R_DLINE;  U.DLINE; 

DISP.RNGE 

=  0., 

DISP.DEFL 

=  3000., 

DISP.HEADING 

=  90.0, 

!  DISPENSE  HEADING  (SUB'S  INITIAL  HDNG) 

DISP.DELAY 

=  0.5, 

!  DELAY  BETWEEN  SUBS  OUT  OF  DISPENSER 

DISP.SUB.CEP 

=  3.0, 

0,  !  ROUNDS  THAT  USE  THIS  PATTERN  DATA  -1  ALL 

»CIRaE4’,  !  SPIRAL;  CIRCLE;  LINE;  RACETRAK;  DUMB 

.F., 

.T,,  !  ARE  SUBS  POWERED  (  .false.  =  glide  or  dumb) 

1800.,  !  or  END  OF  FOOT  PRINT  CUT  OFF;  TOF 

300. ,  !  SUBMUNITION  ALTITUDE 

100.,  !  METERS/SEC. 

0.,  !  CENTER  OF  SPIRAL  OR  CIRCLE  X  relative 

1500.,  !  CENTER  OF  SPIRAL  OR  CIRCLE  Y  relative 
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START.RADIUS 

SCALE 

NO^SUBS.IN.PAT 

START.ANGLE(1) 

START.TIMECl) 

JOIN.ANGLE(l) 

JDIN^RADIUS(l) 

FRAC.CIRC 

SUB^SEPARATIOM 

SPIRAL^MOVE 

DISP.OPTIDN 

DISP.RNGE 

DISP.DEFL 

DISP^HEADING 

DISP^DELAY 

$END 


=  1500.,  !  START  POINT  IN  SPIRAL  OR  CIRCLE  FOR  1st  SUB. 

=  1.,  !  Scale  the  circle  size 

=4,  !  #  SUBMUNIIDNS  IN  PATTERN/ROUND 

=  315. ,135. ,-135 . ,-315. ,  I  Circle  :  each  sub 

=  4*-l.,  ! 4*94. 16,  !  Circle  ;  each  sub  -1  for  calculated  time 

=  315. ,135. ,135. ,315. ,  !  Circle  :  each  sub 

=  500. ,500. ,500. ,500. ,  !  Circle  :  each  sub 

=1.0,  !  FRACTION  OF  CIRCLE  COVERED. 

=700.,  !  DIST  BETWEEN  SUCCESSIVE  "RINGS”  IN  SPIRAL  OR  CIRCLE 

=  0,  !  0  -  MOVES  OUTWARD;  1  -  INWARD 

=  'U^DLINE',  !  BIVNORML;  R.RLINE;  U^RLINE ; R.DLINE ;  U.DLINE;  NONE 

=  0.. 

=  1200., 

=90.,  !  DISPENSE  HEADING  (SUB’S  INITIAL  HDNG) 

=0.0,  !  DELAY  BETWEEN  SUBS  OUT  OF  DISPENSER 


/*  TARGET  DATA  INPUT  ♦/ 

HEADING  SPEED  START  ACCEL 


TARGET 

TYPE 

X 

Y 

Z 

(DEG.) 

(K/H) 

TIHE(S) 

TIME 

'TGT1E» 

'PRIIE^ 

2084. 

3910. 

0.0 

45.0 

10. 

0. 

0. 

5. 

*TGT2E» 

’PRIIE' 

5200 

5950 

0.0 

0.0 

0. 

0. 

0. 

5. 

'TGT3E' 

»PRI1E' 

6200 

5950 

0.0 

0.0 

0. 

0. 

0. 

5. 

»TGT4L' 

»PRI2L» 

5900. 

6100. 

0.0 

135.0 

10. 

0. 

0. 

5. 

'TGT6L» 

'PRI2L' 

5900. 

5930. 

0.0 

215.0 

10. 

0. 

0. 

5. 

’TGT6L» 

»PRI2L' 

6100. 

6100. 

0.0 

0.0 

0. 

0. 

0. 

5. 

'TGT7L» 

'PRI2L* 

5100. 

6100. 

0.0 

0.0 

0. 

0. 

0. 

5. 

’TGT8L» 

'PRI2L' 

6100. 

5940. 

0.0 

0.0 

0. 

0. 

0. 

5. 

»TGT9A’ 

’PRI3A' 

6000. 

6120. 

0.0 

315.0 

10. 

0. 

0. 

5. 

'TGTIOA' 

»PRI3A» 

5800 

6050 

0.0  0, 

.0 

o 

o 

.  0. 

5. 

LENGTH 

WIDTH 

TYPE  R 

OR  F 

(M) 

(M) 

#  ALT  TYPES 

TRUE' 

'R' 

0.1 

0.1 

0 

TRI2L' 

'R' 

0.1 

0.1 

0 

TRI3A' 

'F' 

0.1 

0.1 

0 

^CLUTTER' 

'F' 

0.1  0.1 

0 
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A.1.2  Flight  Path  Input  File  for  8  Submunitions. 

LINE  !  Pattern  Name 

0  I  file  contains  ;  0  -  #  subs  per  round;  1  -  all  subs  in  game 

1,  2,  3,  4,  Munition  # 

200.0  100  90.0  Altitude,  Velocity  ft  Heading(0  along  +X  axis (i. e. .deflection) ) 
RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 
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13200 


GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180.  lend  of  first  sweep 
GO 

RANGE 

13200. 

GO 

RADIUS 

850.  !  Offset  next  sweep  by  0  meters 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 
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GO 

RANGE 


13200 


GO 

RADIUS 

860. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

860. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180.  !  end  of  second  sweep 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 
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“180, 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 
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RADIUS 


850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180.  lend  of  third  sweep 
GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

“180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

“180. 

GO 

RANGE 


13200 


GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180.  lend  of  forth  sweep 
GO 

EXIT 

6,  Munition  # 

200.0  100  0.0  Altitude,  Velocity  ft  Heading (0  along  +X  axis (i.e. .deflection)) 
RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 


A- 10 


850 


theta 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 
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GO 

RADIUS 

850. 

THETA 

-180.  lend  of  first  sveep 
GO 

EXIT 

6,  Munition  # 

200.0  100  90.0  Altitude,  Velocity  k  HeadingCO  along  +X  axis (i.e. .deflection) ) 
RANGE 

200. 

GO 

RADIUS 

270. 

THETA 

-90. 

GO 

RANGE 

12600. 

GO 

RADIUS 

860. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

860. 
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THETA 


-180. 


GO 

RANGE 

13200. 


GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200 

GO 

RADIUS 

850. 

THETA 

- 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180.  lend  of  first  sweep 
GO 

EXIT 

7,  Munition  # 

200.0  100  90.0  Altitude,  Velocity  A  HeadingCO  along  +X  axis (i .e. , deflection) ) 
RANGE 

600. 

GO 

RADIUS 

270. 

THETA 

-90. 

GO 
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RANGE 


12050. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

860. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 


A- 14 


-180 


60 

RANGE 

13200. 

GO 

RADIUS 

860. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180.  tend  of  first  sweep 
GO 

EXIT 

8,  Munition  # 

200.0  100  90.0  Altitude,  Velocity  ft  HeadingCO  along  +X  axis (i.e. .deflection) ) 
RANGE 

1000. 

GO 

RADIUS 

270. 

THETA 

-90. 

GO 

RANGE 

11600. 

60 

RADIUS 

850. 

THETA 

180. 

60 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 
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13200 


GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

180. 

GO 

RANGE 

13200. 

GO 

RADIUS 

850. 

THETA 

-180.  lend  of  first  sweep 
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GO 

EXIT 
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A. 2  Output  Files 

A.2.1  Main  Output  File  for  Example  of  10  (instead  of  200)  Monte  Carlo  Runs. 


^i^t‘*r¥*it**************************************************************************'^*********** 

POWERED  SUBMUNITION  MODEL,  VERSION  4A 

SUBMUNITION  INPUT  FILE  IS:  thesis. in 
SUBMUNITION  SEARCH  PATTERN  FILE  IS:  8sub.cmd 
RUN  BEGAN:  Tus  Feb  13  14:57:12  2001 

see  doe  Test  tl 

RANDOM  SEED:  012342 

DISPENSER  RELIABILITY:  1.00 

TARGET  LOCATION  ERROR  CEP  (METERS):  100.00 

DISPENSER  CEP  (METERS):  13.00 

DISPENSER  PRECISION  ERROR:  1,00  ,  1,00 

PCLOS:  1.00 

INGRESS  TIME  (SECONDS):  0.00 

AIM  POINT  (X):  0.00 

AIM  POINT  (Y):  0.00 

NUMBER  OF  TARGETS:  10 

RANDOM  TARGETS?:  T 

TARGET  AREA  XMAX:  6000.00 

TARGET  AREA  XMIN:  7000.00 

TARGET  AREA  YMAX:  5000.00 

TARGET  AREA  YMIN:  7000.00 

NUMBER  OF  TARGETS:  10 

Value  for  Dans  Variable  is  1.000000 

RANGEPAR  »  0,1000000  TOFPAR  «  0.7100000  PRIOR? AR  »  0.4800000 

NO.ENG^PAR  ■  0.1000000 

Communications  Max  Range  »  9806.600 

FOR  CURRENT  SUB  VEL  k  SCAN  TIME.  OVERLAP  IS  :  38.54764 

SUBMUNITION  RELIABILITY:  0.96 
SEEKER  FOOT  PRINT  WIDTH  (METERS):  600.0 
SEEKER  BEAM  WIDTH:  3.4 
SEEKER  DEPRESSION  ANGLE:  13.0 
SEEKER  SCAN  TIME:  1.800 
SEEKER  FLYBACK  TIME:  0.200 
MINIMUM  TURN  RADIUS  (METERS):  270.00 
COOPERATIVE  ENGAGEMENT  IS  NOT  IMPLEMENTED 
COUNTER  LOGIC  IS:  1  .  1 

SUBMUNITION  TOTAL  SEARCH  TIME:  1200.00 
SUBMUNITION  SEARCH  ALTITUDE  (METERS):  200.0 
SUBMUNITION  SEARCH  VELOCITY  (M/S):  100.0 
NUMBER  OF  SUBMUNITIONS  IN  PATTERN:  8 
SUBMUNITION  DISPENSE  DEFLECTION  (METERS):  3000.00 
SUBMUNITION  INITIAL  HEADING  (DECS):  90.00 
SUBMUNITION  DISPENSE  DELAY(SECS):  0.60 
SUBMUNITION  DISPENSE  CEP  (METERS):  3.00 
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TARGET 

TYPE 

X 

Y 

TGTIE 

PRIIE 

2084.00 

3910.00 

TGT2E 

PRIIE 

6200,00 

5950.00 

TGT3E 

PRIIE 

6200,00 

5960.00 

TGT4L 

PRI2L 

5900.00 

6100.00 

TGT5L 

PRI2L 

6900.00 

5930.00 

TGT6L 

PRI2L 

6100.00 

6100.00 

TGT7L 

PRI2L 

6100.00 

6100.00 

TGT8L 

PRI2L 

6100.00 

6940.00 

TGT9A 

PRI3A 

6000.00 

6120.00 

TGTIOA 

PRI3A 

5800.00 

6060.00 

PRIIE 

R  0.10 

0.10 

PRI2L 

R  0,10 

0.10 

PRI3A 

F  0.10 

0.10 

Z 

HDNG 

SPEED 

START 

TIMES  ACCEL 

TIME 

0.00 

46.0 

10.0 

0.0 

0.0 

6.0 

0.00 

0.0 

0.0 

0.0 

0.0 

6.0 

0.00 

0.0 

0.0 

0.0 

0.0 

6.0 

0.00 

135.0 

10.0 

0.0 

0.0 

6.0 

0.00 

216.0 

10.0 

0.0 

0.0 

6.0 

0.00 

0.0 

0.0 

0.0 

0.0 

6.0 

0.00 

0.0 

0.0 

0.0 

0.0 

6.0 

0.00 

0.0 

0.0 

0.0 

0.0 

6.0 

0.00 

316.0 

10.0 

0.0 

0.0 

6.0 

0.00 

0.0 

0.0 

0.0 

0.0 

6.0 

i^4t^:^ttii^itt4iiti4i******^*i¥************************>*f************************************************* 


REP  NO  ■  1 


♦♦♦♦  COOP 

ENGAGEMENT  (DECISION  RULE) 

SUMMARY 

RANGE 

RATE 

TIME 

PRIORITY 

NO.: 

ENGAGE 

(THRESHOLD  = 

SUB  i 

PARAM 

NORM 

PARAM 

NORM 

PARAM 

NORM 

PARAM 

NORM 

DECISION 

5 

0.10 

0.9764 

0.71 

0.6964 

0.48 

1.0000 

0.10 

0.0000 

1.00 

2 

0.10 

0.9843 

0.71 

0.6968 

0.48 

1.0000 

0.10 

0.0000 

1.00 

1 

0.10 

0.9806 

0.71 

0.7364 

0.48 

1.0000 

0.10 

1.0000 

1.00 

♦♦♦♦  SMART  SUB  SUMMARY  ♦*♦♦ 


REL 

SUB#  RND«  FAIL  tCUTS  iACQ 


#PRI.  ACQ 

ACQ  TGT#  TGTS  SLCTD 
SEL#  SLCTD  SLCTD  ACQ  HI  PRI  HIT  KILL 


tNOT 

ENUF 

FLS  CLT  TIME 


1  1  F  11 

2  1  F  2  2 

3  1  F  2  2 

4  1  F  2  2 

5  1  F  4  4 

6  1  F  2  2 

7  1  F  11 

8  1  F  4  4 


♦♦♦♦  SMART  SUB  LOSS  ♦♦♦♦ 


1  0  2  0  0 

10  3  10 

10  5  10 

10  4  10 

1  0  3  0  0 

1  0  2  0  0 

1  0  105  0  0 

1  0  3  0  0 


T  F  F  F  0 
T  F  F  F  0 
T  T  F  F  0 
T  F  F  F  0 
T  T  F  F  0 
T  T  F  F  0 
F  F  F  T  0 
T  F  F  F  0 


REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


.00) 


A- 19 


0000010220 
0.0  0.0  0.0  0.0  0.0  12.6  0.0  25.0  26.0  0.0 

TOTAL  LOST  :  6  62.6 

DG:  End  of  Rep  1 

DG:  Total  i  of  Reps  ■  10 

^t^4tt^^t^*********’¥’¥**************************************************************************** 

REP  NO  »  2 

****  COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY  ♦♦♦♦ 


RANGE  RATE 

TIME 

PRIORITY 

NO.ENGAGE 

(THRESHOLD  « 

SUB  t 

PARAM  NORM 

PARAM  NORM 

PARAM  NORM 

PARAM  NORM 

DECISION 

♦♦♦♦  SMART  SUB  SUMMARY  ♦♦♦♦ 

#PRI.  ACQ  #NOT 

REL  ACQ  TGT#  TGTS  SLCTD  ENUF 

SUBt  RND#  FAIL  #CUTS  iACQ  SELi  SLCTD  SLCTD  ACQ  HI  PRI  HIT  KILL  FLS  CLT  TIME 


1  1  F  2  2 

2  1  F  3  3 

3  1  F  2  2 

4  1  F  2  2 

6  1  F  11 

6  1  F  3  3 

7  1  F  11 

8  1  F  3  3 


10  6  10 

10  7  10 

1  0  3  0  0 

1  0  2  0  0 

1  0  3  0  0 

10  8  10 

1  0  108  0  0 

10  6  10 


T  T  F  F  0 
T  F  F  F  0 
T  T  F  F  0 
T  F  F  F  0 
T  F  F  F  0 
T  F  F  F  0 
F  F  F  T  0 
T  T  F  F  0 


SMART  SUB  LOSS 

REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


0000010040 
0.0  0.0  0.0  0.0  0.0  12.5  0.0  0.0  60.0  0.0 

TOTAL  LOST  :  6  62.6 

DG;  End  of  Rep  2 

DG:  Total  i  of  Reps  ■  10 

REP  NO  -  3 

♦♦♦♦  COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY  ♦♦♦♦ 


,00) 


A-20 


RANGE  RATE 


TIME 


PRIORITY 


NQ.ENGAGE  (THRESHOLD  *  1.00) 

SUB  #  PARAM  NORM  PARAM  NORM  PARAM  NORM  PARAM  NORM  DECISION 


♦♦♦♦  SMART  SUB  SUMMARY  ♦*** 


SUB# 

RND# 

REL 

FAIL 

#CUTS 

#ACQ 

SEL# 

ACQ 

SLCTD 

TGT# 

SLCTD 

#PRI 

tgts 

ACQ 

ACQ 

SLCTD 

HI  PRI 

HIT 

KILL 

FLS 

CLT 

#NOT 

ENUF 

TIME 

1 

1 

F 

3 

3 

1 

0 

8 

1 

0 

T 

F 

F 

F 

0 

2 

1 

F 

2 

2 

1 

0 

7 

1 

0 

T 

T 

F 

F 

0 

3 

1 

F 

1 

1 

1 

0 

4 

1 

0 

T 

T 

F 

F 

0 

4 

1 

F 

1 

1 

1 

0 

111 

0 

0 

F 

F 

F 

T 

0 

6 

1 

F 

2 

2 

1 

0 

8 

1 

0 

T 

F 

F 

F 

0 

6 

1 

F 

4 

4 

1 

0 

3 

0 

0 

T 

F 

F 

F 

0 

7 

1 

F 

1 

1 

1 

0 

102 

0 

0 

F 

F 

F 

T 

0 

8 

1 

F 

1 

1 

1 

0 

6 

1 

0 

T 

F 

F 

F 

0 

SMART  SUB  LOSS 

REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


0000020040 
0.0  0.0  0.0  0.0  0.0  26.0  0.0  0.0  60.0  0.0 

TOTAL  LOST  :  6  76.0 

DG:  End  of  Rep  3 

DG:  Total  t  of  Reps  *  10 

REP  NO  -  4 

****  COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY  ♦♦♦♦ 

RANGE  RATE  TIME 

SUB  «  PARAM  NORM  PARAM  NORM 

3  0.10  0.9876  0.71  0.6946 


SMART  SUB  SUMMARY  ♦♦♦♦ 


#PRI .  ACQ  #NaT 

REL  ACQ  TGT#  TGTS  SLCTD  ENUF 

SUB#  RND#  FAIL  #CUTS  #ACQ  SEL#  SLCTD  SLCTD  ACQ  HI  PRI  HIT  KILL  FLS  CLT  TIME 

IIF  2  210  810  TFFF0 

21F  2  210  610  TFFF0 

31F  4  410  200  TFFF0 


A-21 


PRIORITY  NO_ENGAGE  (THRESHOLD  »  1.00) 

PARAM  NORM  PARAM  NORM  DECISION 

0.48  1.0000  0.10  0.0000  1.00 


41F  4  410  200  TFFFO 

61F  4  410  610  TFFFO 

61F  2  210  200  TTFFO 

71F  3  310  300  TFFFO 

81F  3  310  610  TTFF0 


SMART  SUB  LOSS  ♦♦♦♦ 

REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


0000000240 
0.0  0.0  0.0  0.0  0.0  0.0  0.0  26.0  60.0  0.0 

TOTAL  LOST  :  6  76,0 

DG:  End  of  Rep  4 

DG:  Total  t  of  Reps  «  10 

REP  NO  “  6 

COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY 

RANGE  RATE  TIME  PRIORITY  NO.ENGAGE  (THRESHOLD  *  1.00) 

SUB  «  PARAM  NORM  PARAM  NORM  PARAM  NORM  PARAM  NORM  DECISION 


SMART  SUB  SUMMARY 


SUB# 

RND« 

REL 

FAIL 

tCUTS 

«Acq 

SEL« 

ACQ 

SLCTD 

TGT# 

SLCTD 

«PRI 

TGTS 

ACQ 

ACQ 

SLCTD 

HI  PRI 

HIT 

KILL 

FLS 

CLT 

#NOT 

ENUF 

TIME 

1 

1 

F 

6 

6 

1 

0 

101 

0 

0 

F 

F 

F 

T 

0 

2 

1 

T 

0 

0 

1 

0 

0 

0 

0 

F 

F 

F 

F 

0 

3 

1 

F 

2 

2 

1 

0 

8 

1 

0 

T 

T 

F 

F 

0 

4 

1 

F 

6 

6 

1 

0 

0 

1 

0 

F 

F 

F 

F 

0 

6 

1 

F 

6 

6 

1 

0 

2 

0 

0 

T 

F 

F 

F 

0 

6 

1 

F 

3 

3 

1 

0 

7 

1 

0 

T 

F 

F 

F 

0 

7 

1 

F 

1 

1 

1 

0 

106 

0 

0 

F 

F 

F 

T 

0 

8 

1 

F 

1 

1 

1 

0 

8 

1 

0 

T 

F 

F 

F 

0 

SMART  SUB  LOSS 

REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


1001020030 
12.6  0.0  0.0  12.6  0.0  26.0  0.0  0.0  37.6  0.0 


A-22 


TOTAL  LOST 


7  87.6 


DG:  End  of  Rep  6 

DG;  Total  i  of  Reps  »  10 

♦♦*♦♦♦*♦♦♦♦*♦♦*♦♦♦♦♦♦♦*♦*♦♦♦♦♦*♦♦*♦♦♦♦♦♦****♦♦♦♦♦********♦♦♦♦♦*♦**♦♦♦♦♦♦♦******♦♦*♦♦♦***♦♦♦♦* 

REP  NO  >  6 

♦♦♦♦  COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY  ♦♦♦♦ 

RANGE  RATE  TIME  PRIORITY  NO.ENGAGE  (THRESHOLD  -  1.00) 

SUB  i  PARAM  NORM  PARAM  NORM  PARAM  NORM  PARAM  NORM  DECISION 


♦♦♦♦  SMART  SUB  SUMMARY  ♦♦♦♦ 

#PRI.  ACQ  iNQT 

REL  ACQ  TGT#  TGTS  SLCTD  ENUF 

SUB#  RND#  FAIL  iCUTS  iACQ  SEL#  SLCTD  SLCTD  ACQ  HI  PRI  HIT  KILL  FLS  CLT  TIME 


IIF  6  610  610  TFFF0 

21F  2  210  710  TTFF0 

31F  1  110  610  TFFFO 

41F  1  110  410  TFFFO 

61F  4  410  710  TFFFO 

61F  2  210  10  00  FFTFO 

71F  1  110  810  TFFFO 

81F  3  310  103  00  FFFTO 


****  SMART  SUB  LOSS  **** 

REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


0000110060 
0.0  0.0  0.0  0.0  12,6  12.6  0.0  0.0  62.6  0.0 

TOTAL  LOST  :  7  87.6 

DG:  End  of  Rep  6 

DG:  Total  #  of  Reps  »  10 

REP  NO  »  7 

*♦♦♦  COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY 

RANGE  RATE  TIME  PRIORITY  NO.ENGAGE  (THRESHOLD  =  1.00) 

SUB  #  PARAM  NORM  PARAM  NORM  PARAM  NORM  PARAM  NORM  DECISION 

3  0,10  0.9684  0.71  0.6962  0.48  1.0000  0.10  0.0000  1.00 


A-23 


♦♦♦♦  SMART  SUB  SUMMARY  ♦♦♦♦ 


SUB# 

RND# 

REL 

FAIL 

»CUTS  #ACQ 

SEL# 

Acq 

SLCTD 

TGT# 

SLCTD 

#PRI. 

TGTS 

Acq 

.  Acq 

SLCTD 

HI  PRI 

HIT 

KILL 

FLS 

CLT 

#NOT 

ENUF 

TIME 

1 

1 

F 

1 

1 

1 

0 

102 

0 

0 

F 

F 

F 

T 

0 

2 

1 

F 

3 

3 

1 

0 

8 

1 

0 

T 

F 

F 

F 

0 

3 

1 

F 

4 

4 

1 

0 

2 

0 

0 

T 

F 

F 

F 

0 

4 

1 

F 

1 

1 

1 

0 

3 

0 

0 

T 

F 

F 

F 

0 

6 

1 

F 

6 

6 

1 

0 

8 

1 

0 

T 

T 

F 

F 

0 

6 

1 

F 

3 

3 

1 

0 

6 

1 

0 

T 

T 

F 

F 

0 

7 

1 

F 

1 

1 

1 

0 

2 

0 

0 

T 

T 

F 

F 

0 

8 

1 

F 

1 

1 

1 

0 

5 

1 

0 

T 

F 

F 

F 

0 

♦♦♦♦  SMART  SUB  LOSS  ***♦ 

REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


0000010220 
0,0  0.0  0.0  0.0  0.0  12.6  0.0  26.0  26.0  0.0 

TOTAL  LOST  :  6  62.6 

DG:  End  of  Rep  7 

DG:  Total  i  of  Reps  «  10 

REP  NO  -  8 

*♦♦♦  COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY  **** 

RANGE  RATE  TIME  PRIORITY  NO.ENGAGE  (THRESHOLD  »  1.00) 

SUB  t  PARAM  NORM  PARAM  NORM  PARAM  NORM  PARAM  NORM  DECISION 


SMART  SUB  SUMMARY  ♦♦♦* 

iPRI.  ACq  #NOT 

REL  ACQ  TGT#  TGTS  SLCTD  ENUF 

SUB#  RND#  FAIL  #CUTS  #ACQ  SEL#  SLCTD  SLCTD  ACQ  HI  PRI  HIT  KILL  FLS  CLT  TIME 


IIF  1  110  710  TTFFO 

21F  2  210  200  TFFFO 

31F  1  110  107  00  FFFTO 

41F  3  310  810  TTFFO 

51F  1  110  200  TTFFO 

61F  1  110  610  T  FFFO 


A-24 


71F  4  410  710  TFFFO 

81F  4  410  410  TFFFO 


♦♦♦♦  SMART  SUB  LOSS 

REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


0000010130 
0.0  0.0  0.0  0.0  0.0  12.6  0.0  12.6  37.6  0.0 

TOTAL  LOST  :  6  62.6 

DG:  End  of  Rep  8 

DG:  Total  #  of  Reps  »  10 

^^i*^^^i^Hi^i^***^*i^************4ii^*************************************************************** 

REP  NO  >  9 

♦♦♦♦  COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY 

RANGE  RATE  TIME  PRIORITY  NO.ENGAGE  (THRESHOLD  «  1.00) 


SUB  i 

PARAM 

NORM 

PARAM 

NORM 

PARAM 

NORM 

PARAM 

NORM 

DECISION 

3 

0.10 

0.9913 

0,71 

0.6946 

0.48 

1.0000 

0.10 

0.0000 

1,00 

6 

0.10 

0.9249 

0.71 

0.6025 

0.48 

1.0000 

0.10 

0.0000 

1.00 

♦♦♦♦  SMART  SUB  SUMMARY  ♦♦♦* 


SUBi 

RND# 

REL 

FAIL 

#CUTS 

tACQ 

SEL# 

ACQ 

SLCTD 

TGT# 

SLCTD 

#PRI 

TGTS 

ACQ 

ACQ 

SLCTD 

HI  PRI 

HIT 

KILL 

FLS 

CLT 

#NOT 

ENUF 

TIME 

1 

1 

F 

1 

1 

1 

0 

113 

0 

0 

F 

F 

F 

T 

0 

2 

1 

F 

3 

3 

1 

0 

6 

1 

0 

T 

F 

F 

F 

0 

3 

1 

F 

6 

5 

1 

0 

8 

1 

0 

T 

F 

F 

F 

0 

4 

1 

T 

0 

0 

1 

0 

0 

0 

0 

F 

F 

F 

F 

0 

6 

1 

F 

2 

2 

1 

0 

3 

0 

0 

T 

F 

F 

F 

0 

6 

1 

F 

4 

4 

1 

0 

3 

0 

0 

T 

F 

F 

F 

0 

7 

1 

F 

4 

3 

1 

0 

6 

1 

0 

T 

F 

F 

F 

0 

8 

1 

F 

3 

3 

1 

0 

8 

1 

0 

T 

T 

F 

F 

0 

♦♦♦♦  SMART  SUB  LOSS  ♦♦♦♦ 

REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


1000010140 
12.6  0.0  0.0  0.0  0.0  12.6  0.0  12.6  60.0  0.0 

TOTAL  LOST  :  7  87.6 


A-25 


DG:  End  of  Rop 
DG:  Total  t  of  Reps 


9 


10 


♦♦♦*♦♦♦♦♦*♦♦♦♦*♦♦♦»♦♦♦**♦*♦♦♦***♦♦♦♦♦♦♦**♦*♦♦♦**♦♦*♦**♦♦*♦♦♦♦♦♦*♦*♦♦*♦♦*♦****♦*♦♦♦♦♦****♦♦♦** 
REP  NO  ■  10 

♦♦♦*  COOP  ENGAGEMENT  (DECISION  RULE)  SUMMARY 

RANGE  RATE  TIME 

SUB  t  PARAM  NORM  PARAM  NORM 

4  0.10  0.9810  0.71  0.6960 


SMART  SUB  SUMMARY 


SUB# 

RND# 

REL 

FAIL 

#CUTS 

#ACQ 

SEL# 

Acq 

SLCTD 

TGT# 

SLCTD 

#PRI 

TGTS 

ACQ 

ACQ 

SLCTD 

HI  PRI 

HIT 

KILL 

FLS 

CLT 

#NOT 

ENUF 

TIME 

1 

1 

F 

5 

6 

1 

0 

7 

1 

0 

T 

F 

F 

F 

0 

2 

1 

F 

1 

1 

1 

0 

106 

0 

0 

F 

F 

F 

T 

0 

3 

1 

F 

4 

4 

1 

0 

6 

2 

0 

T 

F 

F 

F 

0 

4 

1 

F 

6 

5 

1 

0 

3 

0 

0 

T 

F 

F 

F 

0 

6 

1 

F 

2 

2 

1 

0 

6 

1 

0 

T 

F 

F 

F 

0 

6 

1 

F 

2 

2 

1 

0 

2 

0 

0 

T 

F 

F 

F 

0 

7 

1 

F 

4 

4 

1 

0 

3 

0 

0 

T 

T 

F 

F 

0 

8 

1 

F 

1 

1 

1 

0 

3 

0 

0 

T 

F 

F 

F 

0 

♦♦♦♦  SMART  SUB  LOSS  ♦♦♦♦ 

REL  F  NO  CrUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RND  REL  F 


0000010160 
0.0  0.0  0.0  0.0  0.0  12.6  0.0  12.6  62.6  0.0 

TOTAL  LOST  :  7  87.6 


DG:  End  of  Rep  10 

DG:  Total  t  of  Reps  «  10 

HIT/KILL  FREQ 

4  10 

6  3  0 

7  6  0 

8  10 


♦♦♦♦  SMART  SUB  LOSS  AVG 


PRIORITY  NO .ENGAGE  (THRESHOLD  »  1.00) 

PARAM  NORM  PARAM  NORM  DECISION 

0.48  1.0000  0.10  0.0000  1.00 


A-26 


♦  ♦ 

♦  * 

*  THE  AVERAGE  NUMBER  OF  SUBMUNITIONS  LOST  BY  CAUSE  * 

^  - -  * 


♦  REL  F  NO  CUT  NO  ACQ  NO  SEL  FLSTGT  CLTTGT  NO  HIT  DEDTGT  NO  KIL  RFAIL  ♦ 

♦  AVG  0.200  0.000  0.000  0.100  0.100  1.100  0.000  0.900  3.600  0.000  ♦ 

♦  TOT  2.600  0.000  0.000  1.260  1.260  13.760  0.000  11.260  46.000  0.000  ♦ 

«  * 

♦  TOTAL  LOST  :  6.0  76.0  * 

«  * 

♦  * 
♦♦♦♦♦♦♦♦♦♦*♦♦♦♦♦**♦♦♦♦♦♦♦*♦♦♦♦♦♦♦♦♦*♦*♦♦♦♦♦♦♦♦**♦♦♦♦♦♦♦♦♦♦*♦**♦*♦****♦**♦♦♦*♦*******♦**♦♦**** 


4< 

•  ACQUISITIONS 

t  SELECTIONS 

«  HITS 

«  KILLS 

»  UNIQUE  KILLS 

4c 

TARGET 

MEAN  STD  DEV 

MEAN 

STD  DEV 

MEAN 

STD  DEV 

MEAN 

STD  DEV 

MEAN 

STD  DEV 

4c 

* 

* 

* 

1  TGTIE 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

4c 

4c 

* 

2  TGT2E 

1.60 

1.18 

1.20 

1.03 

1.20 

1.03 

0.60 

0.84 

0.40 

0.62 

4> 

4c 

3  TGT3E 

1.40 

1.07 

1.30 

1.16 

1.30 

1.16 

0.40 

0.70 

0.30 

0.48 

4c 

4c 

4  TGT4L 

0.60 

0.63 

0.40 

0.62 

0.40 

0.62 

0.10 

0.32 

0.10 

0.32 

4c 

4c 

5  TGT6L 

0.90 

0.67 

0.80 

0.63 

0.80 

0.63 

0.20 

0,42 

0.20 

0.42 

4c 

4c 

6  TGT6L 

0.80 

0.42 

0.80 

0.42 

0.80 

0.42 

0,30 

0.48 

0.30 

0.48 

4c 

4c 

7  TGT7L 

0.90 

0.74 

0.80 

0.79 

0.80 

0.79 

0.30 

0.48 

0.30 

0.48 

4c 

4c 

8  TGT8L 

1.20 

0.79 

1.20 

0.79 

1.20 

0.79 

0.40 

0.62 

0.40 

0.62 

4c 

4c 

9  TGT9A 

1.10 

1.10 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

4c 

4c 

* 

10  TGTIOA  1.00 

0.94 

0.10 

0.32 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

4c 

4c 

* 

* 

TOTAL  i 

OF  UNIQUE 

KILLS 

WAS  : 

2.00 

4c 

* 

♦  * 

♦  * 


♦♦♦****♦♦♦♦♦♦♦****♦*♦♦♦♦♦♦**♦♦♦♦♦♦♦♦♦♦♦*♦♦♦♦*♦♦♦♦♦♦♦♦***♦♦♦♦*♦♦****♦♦*♦****♦♦*♦♦♦******♦*♦*** 

«  * 

♦  «  ACQUISITIONS  «  SELECTIONS  #  HITS  #  KILLS  #  UNIQUE  KILLS  ♦ 

♦  TARGET  MEAN  STD  DEV  MEAN  STD  DEV  MEAN  STD  DEV  MEAN  STD  DEV  MEAN  STD  DEV  ♦ 

♦  . . . . . . . .  ♦ 

4c  * 

♦  PRIIE  2.900  1.792  2.500  1.681  2.600  1.681  1.000  0,943  0.700  0.676  * 


♦  PRI2L  4.300  0.823  4.000  1.165  4.000  1.166  1.300  0.676  1.300  0.676  ♦ 

♦  PRI3A  2.100  1.792  0.100  0.316  0.000  0.000  0.000  0.000  0.000  0.000  ♦ 

♦  * 

*  * 

*  TOTAL  «  OF  UNIQUE  KILLS  WAS  :  2.00  * 

*  TOTAL  t  OF  KILLS  WAS  :  2.30  * 

*  * 


TOTAL 

NUMBER 

OF 

ROUND 

FAILURES  : 

0 

OUT 

OF  : 

10 

RES. 

REL. 

:  1.00 

TOTAL 

NUMBER 

OF 

SUB 

FAILURES  : 

2 

OUT 

OF  : 

78 

RES.  REL, 

:  0.97 

TOTAL 

NUMBER 

OF 

PK 

FAILURES  : 

42 

OUT 

OF  : 

66 

RES. 

PK 

:  0.36 

RUN  FINISHED:  Tue  Feb  13  14:67:20  2001 

*♦**♦♦♦♦♦♦♦♦♦*♦*♦*♦♦♦♦*♦♦♦*♦*♦♦***♦♦*****♦♦♦♦♦♦♦***♦♦♦****♦♦★***♦*♦♦*♦♦♦♦♦♦♦*♦**♦♦♦*♦♦***♦*★♦ 
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A.2.2  History  Output  File  for  1  Repetition 


TIME 

(SEC.) 

1— . 

• 

TYPENAME  X 

- 1  |.. 

Y 

-SUBl 

« 

WN— 

TYPEH. 

0.600 

0 

-9999.00 

-9999.00 

1 

SUB 

1.000 

0 

-9999.00 

-9909.00 

2 

SUB 

1.600 

0 

-9999.00 

-9999.00 

3 

SUB 

2.000 

0 

-9999.00 

-9999.00 

4 

SUB 

2.600 

0 

-9999.00 

-9999.00 

6 

SUB 

3.000 

0 

-9999.00 

-9999.00 

6 

SUB 

3.600 

0 

-9999.00 

-9999.00 

7 

SUB 

4.000 

0 

-9999.00 

-9999.00 

8 

SUB 

84.681 

102 

CLUTTER 

-9999.00 

-9909.00 

5 

SUB 

84.682 

102 

CLUTTER 

-9999.00 

-9999.00 

6 

SUB 

84.582 

102 

CLUTTER 

-9999.00 

-9999.00 

5 

SUB 

178.662 

111 

CLUTTER 

-9999.00 

-9999.00 

3 

SUB 

178.663 

111 

CLUTTER 

-9999.00 

-9999.00 

3 

SUB 

178.663 

111 

CLUTTER 

-9999.00 

-9999.00 

3 

SUB 

266.495 

106 

CLU'l  lEH 

-9999.00 

-9999.00 

4 

SUB 

265.496 

106 

CLUTTER 

-9999.00 

-9999.00 

4 

SUB 

266.496 

106 

CLUTTER 

-9999.00 

-9999.00 

4 

SUB 

272.069 

106 

CLl^^ER 

-9999.00 

-9999.00 

8 

SUB 

272.060 

106 

CLUTTER 

-9999.00 

-9999.00 

8 

SUB 

272.060 

105 

CLUTTER 

-9999.00 

-9999.00 

8 

SUB 

273.129 

106 

CLUTTER 

-9999.00 

-9999.00 

7 

SUB 

273.130 

106 

CLUTTER 

-9999.00 

-9999.00 

7 

SUB 

273.130 

105 

CLUTTER 

-9999.00 

-9999.00 

7 

SUB 

273.130 

106 

CLUTTER 

-9999.00 

-9999.00 

7 

SUB 

274.129 

106 

CLUTTER 

-9999.00 

-9999.00 

7 

SUB 

274.129 

106 

CLUTTER 

-9999.00 

-9999.00 

7 

SUB 

412.920 

107 

CLUTTER 

-9999.00 

-9999.00 

8 

SUB 

412.921 

107 

CLUTTER 

-9999.00 

-9999.00 

8 

SUB 

412.921 

107 

CLUTTER 

-9999.00 

-9999.00 

8 

SUB 

426.463 

112 

CLUTTER 

-9999.00 

-9999.00 

6 

SUB 

426.464 

112 

CLUTTER 

-9999.00 

-9999.00 

6 

SUB 

426.464 

112 

CLUTTER 

-9999.00 

-9999.00 

6 

SUB 

606.498 

113 

CLUTTER 

-9999.00 

-9999.00 

8 

SUB 

506.499 

113 

CLUllER 

-9999.00 

-9999.00 

8 

SUB 

606.499 

113 

CLU 1  icR 

-9999.00 

-9999.00 

8 

SUB 

629.268 

116 

CLUTTER 

-9999.00 

-9999.00 

6 

SUB 

629.269 

116 

CLUTTER 

-9999.00 

-9999.00 

6 

SUB 

629.269 

116 

CLUTTER 

-9999.00 

-9999.00 

5 

SUB 

636.908 

10 

PRI3A 

6641.49 

5472.88 

6 

SUB 

636.909 

10 

PRI3A 

6641.49 

6472.88 

5 

SUB 

636.909 

10 

PRI3A 

6641.49 

6472.88 

6 

SUB 

636.909 

10 

PRI3A 

6641.49 

6472.88 

5 

SUB 

638,713 

3 

PRIIE 

6391.06 

6667.98 

8 

SUB 

638.714 

3 

PRIIE 

6391.06 

6667.98 

8 

SUB 

638.714 

3 

PRIIE 

6391.06 

6667.98 

8 

SUB 

638.714 

3 

PRIIE 

6391.06 

6667.98 

8 

SUB 

638.919 

3 

PRIIE 

6391.06 

6667.98 

8 

SUB 

538.919 

3 

PRIIE 

6391.06 

6567.98 

8 

SUB 

638.919 

3 

PRIIE 

6391.06 

6567.98 

8 

SUB 

639.761 

2 

PRIIE 

6248.67 

6776.71 

6 

SUB 

639.762 

2 

PRIIE 

6248.57 

5776.71 

6 

SUB 

639.762 

2 

PRIIE 

6248.67 

6776.71 

6 

SUB 

539.762 

2 

PRIIE 

6248.57 

6776.71 

6 

SUB 

639.945 

2 

PRIIE 

6248.67 

6775.71 

6 

SUB 

539.946 

2 

PRIIE 

6248.57 

6776.71 

6 

SUB 

639.946 

2 

PRIIE 

6248.67 

6776.71 

6 

SUB 

640.469 

4 

PRX2L 

4909.72 

6261.76 

4 

SUB 

640.470 

4 

PRI2L 

4909.72 

6261.76 

4 

SUB 

640.470 

4 

PRI2L 

4909.72 

6261.76 

4 

SUB 

640.470 

4 

PRI2L 

4909.72 

6261.75 

4 

SUB 

640.673 

4 

PRI2L 

4909.32 

6262.15 

4 

SUB 

640.673 

4 

PRI2L 

4909.32 

6262.15 

4 

SUB 

640.673 

4 

PRI2L 

4909.32 

6262.16 

4 

SUB 

546.884 

6 

PRI2L 

4492.66 

6737.46 

3 

SUB 

I . ID . I  BVEHl  I . THREAT  COORDINATES - 1 

«  TYPENAME  DESCRIPTION  I  Y  Z 


Q  - SUB;  DISPENSED  -1564.22  219.73  200.00 

0  -  3UB;  DISPENSED  -1130.60  215.13  200.00 

0  -  SUB:  DISPENSED  -704.78  211.80  200.00 

0  - —  SUB:  dispensed  -273.17  216.86  200.00 

0  -  SUB;  DISPENSED  162.56  216.82  200.00 

0  -  SUB:  DISPENSED  684.37  219.36  200.00 

0  -  SUB:  DISPENSED  1006,76  216.38  200,00 

0  -  SUB:  DISPENSED  1442.70  217.98  200.00 

0  - TCT:  entered  FOV  8360.62  216.82  200.00 

0  -  TGT:  ACQUIRED  8360.72  216.82  200.00 

4  CLUTTER  TCT:  ID  CORRECT  8360.72  216.82  200.00 

0  -  TGT;  ENTERED  FOV  996.22  11565.96  200.00 

0  -  TGT:  acquired  995.22  11666.85  200.00 

4  CLUTTER  TGT:  ID  CORRECT  996.22  11666.86  200.00 

0  -  TGT;  ENTERED  FOV  1426.83  3937.70  200.00 

G  -  TGT;  ACQUIRED  1426.83  3937.60  200.00 

4  CLUTTER  TGT:  ID  CORRECT  1426.83  3937.60  200.00 

Q  -  TGT:  ENTERED  FOV  2201.23  3187.98  200.00 

0  -  TGT:  ACQUIRED  2201.13  3187.98  200.00 

4  CLUTTER  TGT:  ID  CORRECT  2201.13  3187.98  200.00 

Q  -  TGT:  ENTERED  FOV  2108.36  2786.38  200.00 

G  -  TGT:  ACQUIRED  2108.26  2786.38  200.00 

2  PRI2L  TCT:  ID  INCORRECT  2108.26  2786.38  200.00 

G  -  TGT:  PRIORITY  ACQUIRED  2108.26  2786.38  200.00 

G  -  TGT:  SELECTED  2008.36  2786.38  200.00 

G  - TGT:  SLCTD  HAS  CLUT  TCT  2008.36  2786.38  200.00 

G  -  TGT:  ENTERED  FOV  9439.88  4887.98  200.00 

G  -  TGT;  ACQUIRED  9439.98  4887.98  200.00 

4  CLUTTER  TGT:  ID  CORRECT  9439.98  4887.98  200.00 

G  .  TGT:  ENTERED  FOV  10835.84  4089.36  200.00 

G  -  TGT:  ACQUIRED  10836.94  4089.36  200.00 

4  CLUTTER  TGT:  ID  CORRECT  10835.94  4089.36  200.00 

G  -  TCT:  ENTERED  FOV  10498.04  6687.98  200,00 

G  - TGT:  ACQUIRED  10497.94  6587.98  200.00 

4  CLUTTER  TGT:  ID  CORRECT  10497.94  6687.98  200.00 

0  -  TCT:  ENTERED  FOV  8286.87  6316.82  200,00 

0  -  TGT:  ACQUIRED  8286.77  5316.82  200.00 

4  CLUTTER  TGT:  ID  CORRECT  8286.77  6316.82  200.00 

G  -  TGT:  ENTERED  FOV  7622,82  5316.82  200.00 

0  —  TGT:  ACQUIRED  7622.72  5316.82  200.00 

3  PRI3A  TGT:  ID  CORRECT  7622.72  6316.82  200.00 

G  -  TGT:  NOT  SLCTD »  PRIOR-99  7622.72  6316.82  200.00 

G  -  TGT:  ENTERED  FOV  7276.62  6587.98  200.00 

G  -  TCT;  ACQUIRED  7276.63  6687.98  200.00 

1  PRIIE  TGT:  ID  CORRECT  7276.63  6687.98  200.00 

0  .  TGT:  PRIORITY  ACQUIRED  7276.63  6687.98  200.00 

G  -  TGT:  SELECTED  7266.94  6687.98  200.00 

G  -  TGT:  HIT  7255.94  6587.98  200.00 

G  -  TGT:  NOT  KILLED  7256.94  6687.98  200.00 

G  -  TGT:  ENTERED  FOV  7213,42  5789.36  200.00 

G  -  TGT:  ACQUIRED  7213.33  6789.36  200.00 

1  PRIIE  TGT:  ID  CORRECT  7213.33  6789.36  200.00 

G  -  TGT:  PRIORITY  ACQUIRED  7213.33  6789.36  200.00 

0  -  TGT:  SELECTED  7195.07  5789.36  200.00 

0  .  TGT:  HIT  7195.07  6789.36  200.00 

G  -  TGT:  KILLED  7195.07  6789.36  200.00 

0  -  TGT:  ENTERED  FOV  4826.83  7181.06  200.00 

G  -  TGT:  ACQUIRED  4826.83  7180.96  200.00 

2  PRI2L  TGT;  ID  CORRECT  4826.83  7180.96  200.00 

0  .  TGT;  PRIORITY  ACQUIRED  4826.83  7180.96  200.00 

G  -  TGT;  SELECTED  4826.83  7160.58  200.00 

0  -  TGT:  HIT  4826.83  7160.58  200.00 

G  -  TGT:  NOT  KILLED  4826.83  7160.68  200.00 

G  -  TGT;  ENTERED  FOV  4396.22  6684.48  200.00 
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645.886 

5 

PRI2L 

4492.66 

6737.46 

3 

SUB 

0 

TGT:  ACQUIRED 

645.885 

6 

PRI2L 

4492.66 

6737,46 

3 

SUB 

2 

PRI2L 

TCT:  ID  CORRBCT 

645.885 

5 

PRI2L 

4492.66 

6737.46 

3 

SUB 

0 

TCT;  PRIORITY  ACQUIRED 

546.065 

6 

PRX2L 

4492.24 

6737.16 

3 

SUB 

0 

TOT:  SELECTED 

546.065 

6 

PRI2I. 

4492.24 

6737.16 

3 

SUB 

0 

TGT:  HIT 

546.065 

5 

PRI2L 

4492.24 

6737.16 

3 

SUB 

0 

TOT:  KILLED 

688.677 

7 

PR12L 

5841.63 

6303.46 

2 

SUB 

0 

TCT;  EHTERED  FOV 

688.678 

7 

PRI2L 

5841.63 

6303.46 

2 

SUB 

0 

TGT:  ACQUIRED 

688.678 

T 

PRI2L 

5841.63 

6303.46 

2 

SUB 

3 

PRI3A 

TGT:  ID  IHCORRECT 

688.578 

7 

PRI2L 

5841.63 

6303.46 

2 

SUB 

0 

TGT:  HOT  SLCTD,  PRIOR-99 

714.600 

0 

-9999.00 

-9999.00 

5 

PRIMARY 

6 

(iAT.t.wn 

SUB:  NOTIFIED  OF  T6TS 

714.500 

0 

-9999.00 

-9999.00 

5 

SUB 

0 

SUB:  REDIRECTED 

716.000 

0 

-9999.00 

-9999.00 

2 

PRIMARY 

2 

ntf.T.itn 

SUB:  NOTIFIED  OF  TGTS 

716.000 

0 

-9999.00 

-9999.00 

2 

SUB 

0 

SUB:  REDIRECTED 

729.223 

3 

PRIIE 

6391.06 

6567.98 

5 

SUB 

0 

TGT:  ENTERED  FOV 

729.224 

3 

PRIIE 

6391.06 

6667.98 

5 

SUB 

0 

TGT:  ACQUIRED 

729.224 

3 

PRIIB 

6391.06 

6567.98 

5 

SUB 

1 

PRIIE 

TGT:  ID  CORRBCT 

729.224 

3 

PRIIE 

6391.06 

6667.98 

5 

SUB 

0 

TGT:  PRIORITY  ACQUIRED 

729.224 

3 

PRIIE 

6391.06 

6667.98 

5 

SUB 

0 

TGT:  SELECTED 

729.224 

3 

PRIIE 

6391.06 

6667.98 

6 

SUB 

0 

TGT:  HIT 

729.224 

3 

PRIIE 

6391.06 

6667.98 

5 

SUB 

0 

TGT:  KILLED 

729.267 

3 

PRIIE 

6391.06 

6667.98 

2 

SUB 

0 

TGT:  ENTERED  FOV 

729.258 

3 

PRIIE 

6391.06 

6667.98 

2 

SUB 

0 

TGT:  ACQUIRED 

729.258 

3 

PRIIE 

6391.06 

6567.98 

2 

SUB 

1 

PRIIE 

TGT:  ID  CORRECT 

729.268 

3 

PRIIE 

6391.06 

6667.98 

2 

SUB 

0 

TGT:  PRIORITY  ACQUIRED 

729.268 

3 

PRIIE 

6391.06 

6667.98 

2 

SUB 

0 

TGT:  SELECTED 

729.268 

3 

PRIIE 

6391.06 

6667.98 

2 

SUB 

0 

TGT:  HIT 

729.258 

3 

PRIIE 

6391.06 

6567.98 

2 

SUB 

0 

TGT:  HIT  ALREADY  DEAD 

729.258 

3 

PRIIE 

6391.06 

6567.98 

2 

SUB 

0 

TGT:  NOT  KILLED 

882.500 

0 

-9909.00 

-9999.00 

1 

PRIMARY 

1 

riT.T.m 

SUB:  NOTIFIED  OP  TOTS 

882.500 

0 

-9999.00 

-9909.00 

1 

SUB 

0 

SUB:  REDIRECTED 

892.880 

2 

PRIIE 

6248.57 

6776.71 

1 

SUB 

0 

TGT:  ENTERED  FOV 

892.880 

2 

PRIIE 

6248.67 

6776.71 

1 

SUB 

0 

TCT:  ACQUIRED 

892.880 

2 

PRIIE 

6248.57 

6776.71 

1 

SUB 

1 

PRIIE 

TGT:  ID  CORRECT 

892.880 

2 

PRIIE 

6248.67 

6776.71 

1 

SUB 

0 

TCT:  PRIORITY  ACQUIRED 

892.880 

2 

PRIIE 

6248.67 

5776.71 

1 

SUB 

0 

TGT:  SELECTED 

892.880 

2 

PRIIE 

6248.67 

6776.71 

1 

SUB 

0 

TGT:  HIT 

892.880 

2 

PRIIE 

6248.57 

6775.71 

1 

SUB 

0 

TGT:  HIT  ALREADY  DEAD 

892.880 

2 

PRIIE 

6248.57 

6776.71 

1 

SUB 

0 

TGT:  NOT  KILLED 

4306.22 

4396.22 

4396.22 

4396.22 

4396.22 

4396.22 

6669.40 

6669.40 

6669.40 

6669.40 
7871.14 
7871.14 

6669.40 

6669.40 
7248.74 

7248.66 

7248.66 

7248.66 

7248.66 

7248.66 

7248.66 

6217.31 

6217.31 

6217.31 

6217.31 

6217.31 

6217.31 

6217.31 

6217.31 

6935.78 

6036.78 

6394.47 

6394.47 

6394.47 

6304.47 

6394.47 

6394.47 

6394.47 

6394.47 


6684.38 

6584.38 

6584.38 

6666.34 

6666.34 

6666.34 
6491.44 

5491.64 

6491.64 

6491.64 
7016.82 
7016.82 
8133.71 
8133.71 

6622.37 

6622.38 

6622.38 

6622.38 

6622.38 

6622.38 

6622.38 
7656.33 

7656.23 

7656.23 

7666.23 

7666.23 

7666.23 

7666.23 

7666.23 
4671.61 
4671 . 61 

4761 . 23 

4761 . 32 

4761.32 

4761 . 32 

4761.32 

4761.32 

4761.32 

4761 . 32 


200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200,00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200,00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 

200.00 
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A.2.3  Cuts  Output  File  for  1  Repetition. 


SUBM  NO  : 

TARQET 

1 

1  ■  ■  ■ 

NO.  NIKS 

X 

T 

H 

TENTER 

ID 

TEIIT 

ID 

TINFOV 

I 

Y 

H 

X 

Y 

Z 

2  T0T2E 

SUBM  NO  : 

TAROST 

6248.67 

6776.71 

2 

LOCATION 

0.00 

802.88 

802.04 

0,06 

6170.27 

6627.37 

1  j _ 

104.51 

6394.47 

4761.23 

200.00 

1— — 

NO.  NAME 

I 

Y 

B 

TENTER 

ID 

TEIIT 

ID 

TINFOV 

I 

Y 

B 

I 

Y 

z 

3  TQT3E 

6301.06 

6667.08 

0.00 

720.26 

720.38 

0.12 

6447.37 

6601.73 

284.90 

6217.31 

7666.33 

200.00 

7  Tom 

6841.63 

6303.46 

0.00 

688.68 

688.76 

0.17 

6887.62 

6360.11 

76.88 

6669.40 

6491.44 

200.00 

SUBM  NO  : 

TARGET 

3 

1 - 

NO.  NAME 

X 

Y 

H 

TENTER 

ID 

TEIIT 

ID 

TINFOV 

I 

Y 

B 

X 

Y 

Z 

6  TOTBL 

4402.66 

6737.46 

216.00 

645.88 

646.07 

0.18 

4613.45 

6716.81 

284.12 

4396.22 

6684.48 

200.00 

111  CLUTTER 

006.76 

10670.40 

0.00 

178.66 

178.81 

0.14 

777.00 

10608.28 

255.88 

996.22 

11666.95 

200.00 

SUBM  NO  : 

TARGET 

4 

1 - 

NO.  NAME 

X 

Y 

H 

TENTER 

ID 

TEIIT 

ID 

TINFOV 

I 

Y 

B 

I 

Y 

Z 

4  TQT4L 

4000.72 

6261.76 

136.00 

640.47 

640.67 

0.21 

5045.06 

6313.39 

284.12 

4826.83 

7181.06 

200.00 

106  CLUTTER 

1247.18 

2086.72 

0.00 

266.60 

266.60 

0.20 

1208.60 

3070.04 

266.88 

1426.83 

3937.70 

200.00 

SUBM  NO  t 

TARGET 

6 

_ 1 

1 

- } 

NO.  NAME 

Z 

Y 

B 

TENTER 

ID 

TEIIT 

ID 

TINFOV 

I 

Y 

I 

Y 

z 

3  T0T3R 

6301.06 

6667.08 

0.00 

720.22 

720.41 

0.10 

6367.28 

6369.11 

180.86 

7248.74 

6622.37 

200.00 

10  TGTIOA 

6641.40 

6472.88 

0.00 

636.01 

636.07 

0.16 

6766.16 

5535.04 

166.88 

7622.82 

6316.82 

200.00 

102  CLUTTER 

0212.46 

386.26 

0.00 

84.68 

84.77 

0.10 

0228.20 

436.04 

14.12 

8360.62 

216.82 

200.00 

lie  CLUTTER 

7208.01 

6300.63 

0.00 

620.27 

620.42 

0.16 

7410.20 

6536.04 

165.88 

8286.87 

6316.82 

200.00 

SUBM  NO  : 

TARGET 

6 

oiTs  pnif 

.11 

^  oim  f  HP 

_ _ _ 1 

1 - 

-  LOCATION 

1  ‘ 

••  S>Ui>  rUf 

I  1  ■  ' 

*  9U1>  I«Uw 

NO.  NAME 

X 

Y 

B 

TENTER 

ID 

TEIIT 

ID 

TINFOV 

X 

Y 

B 

X 

Y 

Z 

2  T0T2B 

6248.67 

6776.71 

0.00 

630.76 

630.06 

0.18 

6346.76 

6671.13 

194.12 

7213.42 

6789.36 

200.00 

112  CLUTTER 

11770.81 

3018.87 

0.00 

426.46 

426.67 

0.20 

11703.60 

3871.13 

-14.12 

10836.84 

4089.36 

200.00 

SUBM  NO  : 

TARGET 

7 

4ifrD  cpnv 

II 

»  eim  1  np 

_ _ _  j 

1 - 

LOCATION 

““  SUo  rUV 

1  j  1.^... 

*  DUD  i«Uw 

NO.  NAME 

X 

Y 

B 

TENTER 

ID 

TEIIT 

ID 

TINFOV 

I 

Y 

I 

Y 

z 

106  CLUTTER 

1247.18 

2086.72 

0.00 

273.13 

1240.70 

2568.15 

194.12 

2108.36 

2786.38 

200.00 

SUBM  NO  : 

TARGET 

8 

1 , 

*-p  QiTR  pnv 

•  fliiB  I  nn 

_ 1 

[ 

1— - - 

*  LOvATIQH 

1 

9UO  rUf 

DUD  JUUU 

NO.  NAME 

X 

Y 

B 

TENTER 

ID 

TEIIT 

ID 

TINFOV 

I 

Y 

B 

I 

Y 

z 

3  T0T3E 

6301.06 

6667.08 

0.00 

638.71 

638.02 

0.21 

6408.96 

6369.76 

194.12 

7276.62 

6687.98 

200.00 

106  CLUTTER 

1247.18 

2086.72 

0.00 

272.06 

272.22 

0.16 

1333.56 

3406.21 

166.88 

2201 . 23 

3187.98 

200.00 

107  CLUTTER 

10337.40 

4857.01 

0.00 

412.02 

413.14 

0.22 

10307.65 

5106.21 

14.12 

9439.88 

4887.98 

200.00 

113  CLUTTER 

0643.71 

6616.06 

0.00 

606.50 

506.68 

0.18 

0630.37 

6806.21 

165.88 

10498.04 

6687.98 

200.00 
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A.2.4  Sample  Playback  Output.  Circles  are  targets  (four  of  which  are  moving) 
Eight  submunitions  are  searching  the  area  in  two  sets  of  four. 


Figure  A.l  Screen  Captmre  of  a  Sample  Playback 
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Appendix  B.  Design  of  Experiments  Files 


B.l  Example  Design 


BBI 

HI 

m 

wmm. 

I 

■ 

^■1 

5 

Block  1  i 

0.80 

0.80: 

0.40 

0.80 

5000.00 

2.42^ 

2.97  ; 

5.805 

7.78 

2I  21 

Block  1  ; 

0.80 

0.40: 

0.80 

0.80 

5000.00 

2.35? 

2.94? 

5.82 

7.96 

31  2 

Block  1  i 

0.40 

0.80? 

0.80 

0.40 

15000.00 

2.15 

3.07 

5.895 

8.005 

4!  15 

Block  1  ; 

0.80 

0.80 : 

0.80 

0.40 

5000.00 

2.16 

2.81 ; 

5.6 

7.68 

s\  17 

Block  1  i 

0.80 

_ 0.80: 

0.40 

0.40 

15000.00 

1 .99 

2.87, 

5.63 

7.715 

ej  22 

Block  1  j 

0.80 

0.40 : 

0.40 

0.80 

15000.00 

2.38: 

3.1 ; 

5.935 

7.955 

7!  16 

Block  1  1 

0.40 

0.40: 

0.80 

0.80 

15000.00 

2.34; 

3.16? 

6.06 

8.425 

e!  11 

Block  1  ! 

0.40 

0.80 ; 

0.40 

0.80 

15000.00 

2.4; 

3.21  ? 

6.11 

8.54 

9  26 

Block  1  i 

0.80 

0.40: 

0.80 

0.40 

15000.00 

2.01  ? 

2.95 

5.76 

8.1 

10  6 

Block  1  1 

0.40 

0.80 : 

0.80 

0.80 

5000.00 

2.47; 

3.1  : 

5.97 

7.985 

11  I  4 

Block  1  i 

0.40 

0.40? 

0.40 

0.40 

5000.00 

2.69? 

3.07  ; 

6.075 

8.35 

12;  19 

Block  1  ; 

0.24 

0.60 ; 

0.60 

0.60 

10000.00 

2.41  : 

2.95 

6.1 

8,67 

13:  13 

Block  1  1 

0.96 

0.60 : 

0.60 

0.60 

10000.00 

215 

2.9 

5.625 

7.725 

14!  1 

Block  1  ! 

0.BO 

0.24? 

0.60 

0.60 

10000.00 

2.39 

3.03 

5  99 

8.25 

15|  8 

Block  1  1 

0.60: 

i  0-96; 

0.60 

0.60 

10000.00 

2.22 

2.93 

5.815 

7.795 

lei  24 

Block  1  i 

0.60 

0.60: 

0.24 

0.60 

10000.00: 

2.49, 

3.09 

6.02 

8.41 

171  9 

Block  1  i 

0.601 

i  0.60; 

0.96 

0.60 

10000.00: 

2.1 6 ; 

2.84: 

5.675 

7.8 

18  i  23 

Block  1  i 

0.60 

0.60: 

0.60 

0.24 

10000.00: 

2.17; 

2.89- 

5.725 

7.94 

19!  20 

Block  1  1 

0.60 

0.60; 

0.60? 

0.96 

10000.00  j 

2.24: 

2.93: 

5.85 

7.89 

20;  12 

Block  1  1 

0.60' 

0.60? 

0.60 

0.60 

894  .20  ' 

2.67 

3.16 

6.05 

8.255 

21  i  3 

Block  1  ; 

0.60 ! 

0.60 : 

0.60 

0.60 

19105.80 

2.26 

3.14 

5.905 

7.8 

22 1  18 

Block  1  1 

0.60' 

1  0.60? 

0.60 

0.60 

10000.00: 

2.25 

2.99 

5.89 

7.955 

231  7 

Block  1  1 

0.60 ; 

1  0.60 : 

0.60 

0.60 

10000.00? 

2.3: 

3.06: 

5.89; 

7.79 

24|  25 

Block  1  ; 

0.60 

1  0.60  ? 

0.60 

0.60 

10000.00' 

2.25 

2.96 

5.88 

7.985 

25i  14 

Block  1 : 

0.60 

1  0.60 ; 

0.60 

0.60 

10000.00 

2.27: 

3.01 : 

5.9 

8 

26;  10 

Block  1 

0.60 

0.60 

0.60 

0.60 

10000.00 

2.25 

2.97 

5.86 

7.905 

B.2  Example  ANOVA  for  Unique  Kills  Response 

Response :  unique  K 

ANOVA  for  Response  Surface  Quadratic  Model 

Analysis  of  variance  table  [Partial,  sum  of  squares] 

Sum  of  Mean  F 

Source  Squares  DF  Square  Value  Prob  >  F 

Model  0.708355  20  0.03542  89.2733  <  0.0001  significant 

A  0.0338  1  0.03380  85.1956  0.0003 

B  0.01445  1  0.01445  36.4224  0.0018 

C  0.05445  1  0.05445  137.2456  <  0.0001 

D  0.00245  1  0.00245  6.1754  0.0555 

E  0.08405  1  0.08405  211.8548  <  0.0001 

A2  0.000305  1  0.00030  0.7681  0.4209 

B2  0.00268  1  0.00268  6.7560  0.0483 

C2  0.006277  1  0.00628  15.8204  0.0106 

D2  0.007309  1  0.00731  18.4221  0.0078 

E2  0.073654  1  0.07365  185.6501  <  0.0001 

AB  0.009365  1  0.00936  23.6050  0.0046 

AC  0.00175  1  0.00175  4.4104  0.0897 

AD  0.001724  1  0.00172  4.3460  0.0915 

AE  0.004176  1  0.00418  10.5252  0.0228 

BC  0.000818  1  0.00082  2.0625  0.2104 

BD  0.003542  1  0.00354  8.9275  0.0305 

BE  0.001436  1  0.00144  3.6185  0.1155 

CD  0.006319  1  0.00632  15.9274  0.0104 

CE  0.000459  1  0.00046  1.1569  0.3312 

DE  0.010082  1  0.01008  25.4115  0.0040 

Residual  0.001984  5  0.00040 

Lack  of  Fit  6.37E-05  1  0.00006  0.1326  0.7341  not  significant 

Pure  Error  0.00192  4  0.00048 

Cor  Total  0.710338  25 

The  Model  F-value  of  89.27  implies  the  model  is  significant.  There  is  only 

a  O.Oiy,  chance  that  a  "Model  F-Value"  this  large  could  occur  due  to  noise. 

Values  of  "Prob  >  F"  less  than  0.0500  indicate  model  terms  are  significant. 

In  this  case  A,  B,  C,  E,  B*2,  C“2,  D“2,  E'‘2,  AB,  AE,  BD,  CD,  DE  are  significant  model  terms. 

Values  greater  than  0.1000  indicate  the  model  terms  are  not  significant. 

If  there  are  many  insignificant  model  terms  (not  counting  those  required  to  support  hierarchy) , 

model  reduction  may  improve  your  model. 


The  "Lack  of  Fit  F-value"  of  0.13  implies  the  Lack  of  Fit  is  not  significant  relative  to  the  pur 

error.  There  is  a  73.41*/,  chance  that  a  "Lack  of  Fit  F-value"  this  large  could  occur  due 

to  noise.  Non-significant  lack  of  fit  is  good  —  we  want  the  model  to  fit, 

Std.  Dev.  0.019918  R-SquaredO . 9972074 

Mean  2.301538  Adj  R-SquO . 9860372 

C.V.  0.865429  Pred  R-SqO. 9314578 

PRESS  0.048688  Adeq  Prec39 . 164955 

The  "Pred  R-Squared"  of  0.9315  is  in  reasonable  agreement  with  the  "Adj  R-Squared"  of  0.9860. 

"Adeq  Precision"  measures  the  signal  to  noise  ratio.  A  ratio  greater  than  4  is  desirable.  Your 

ratio  of  39.165  indicates  an  adequate  signal.  This  model  can  be  used  to  navigate  the  design  spa 


Coefficient 

Standard 

95*/.  Cl 

95*/,  Cl 

Factor 

Estimate 

DF 

Error 

Low 

High 

VIF 

Intercept 

2.265321 

1 

0.0083 

2.2440 

2.2866 

A-RDOT 

-0.07138 

1 

0.0077 

-0.0913 

-0.0515 

2.6525 

B-TOF 

-0.04667 

1 

0.0077 

-0.0666 

-0.0268 

2.6525 

C-Prior 

-0.0906 

1 

0.0077 

-0.1105 

-0.0707 

2.6525 

D-Engage 

0.019219 

1 

0.0077 

-0.0007 

0.0391 

2.6525 

l-Comm  Range-0 . 11257 

1 

0.0077 

-0.1324 

-0.0927 

2.6525 

A2 

0.003834 

1 

0.0044 

-0.0074 

0.0151 

1.0152 

B2 

0.011372 

1 

0.0044 

0.0001 

0.0226 

1.0152 

C2 

0.017402 

1 

0 . 0044 

0.0062 

0.0286 

1.0152 

D2 

-0.01878 

1 

0 . 0044 

-0.0300 

-0.0075 

1.0152 

E2 

0.059614 

1 

0.0044 

0.0484 

0.0709 

1.0152 

AB 

-0.05536 

1 

0.0114 

-0.0846 

-0.0261 

3.5867 

AC 

-0.02393 

1 

0.0114 

-0.0532 

0.0054 

3.5867 

AD 

0.023752 

1 

0.0114 

-0.0055 

0.0530 

3.5867 

AE 

-0.03696 

1 

0.0114 

-0.0663 

-0.0077 

3.5867 

BC 

0.016363 

1 

0.0114 

-0.0129 

0.0457 

3.5867 

BD 

0.034043 

1 

0.0114 

0.0048 

0.0633 

3 . 5867 

BE 

-0.02167 

1 

0.0114 

-0.0510 

0.0076 

3.5867 

CD 

0.045471 

1 

0.0114 

0.0162 

0.0748 

3.5867 

CE 

0.012255 

1 

0.0114 

-0.0170 

0.0415 

3.5867 

DE 

0.057435 

1 

0.0114 

0.0281 

0.0867 

3.5867 

Final  Equation  in  Terms  of  Coded  Factors: 
unique  K  = 
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2.265321 
-0.07138  *  A 
-0.04667  *  B 
-0.0906  *  C 
0.019219  *  D 
-0.11257  ♦  E 
0.003834  *  A2 
0.011372  *  B2 
0.017402  *  C2 
-0.01878  *  D2 
0.059614  *  E2 
-0.05536  *  A  ♦  B 
-0.02393  ♦  A  *  C 
0.023752  *  A  *  D 
-0.03696  *  A  *  E 
0.016363  *  B  ♦  C 
0.034043  *  B  ♦  D 
-0.02167  *  B  ♦  E 
0.045471  *  C  *  D 
0.012255  ♦  C  ♦  E 
0.057435  ♦  D  *  E 


Final  Equation  in  Terms  of  Actual  Factors: 


unique  K 

= 

3.851125 

0.730653 

ROOT 

-0.28355 

* 

TOF 

-1.66622 

♦ 

Prior 

-1.46388 

♦ 

Engage 

-7.7E-05 

* 

Comm  Range 

0.095861 

* 

RD0T2 

0.284305 

* 

T0F2 

0.435061 

* 

Prior2 

-0.46947 

* 

Engage2 

2.38E-09 

* 

Comm  Range2 

-1.38389 

* 

ROOT  ♦  TOF 

-0.59819 

* 

ROOT  ♦  Prior 

0.593807 

* 

ROOT  *  Engage 

-3.7E-05 

* 

ROOT  ♦  Comm  Range 

0.409071 

* 

TOF  *  Prior 

0.851069 

* 

TOF  *  Engage 
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‘-2.2E-05  *  TOP  ♦  Comm  Range 
1.13677  *  Prior  *  Engage 
1.23E-05  ★  Prior  ♦  Comm  Range 
5.74E-05  *  Engage  *  Comm  Range 


Standard 
Order 
1 
2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24  2.25 

25  2.27 

26  2.25  2.26532  -0.01532 


0.98153  -0.40061 

0.40615 

-0.36421 

5 

0.98153  -0.40061 

0.40615 

-0.36421 

21 

0.98153  -0.40061 

0.40615 

-0.36421 

2 

0.98153  -0.40061 

0.40615 

-0.36421 

15 

0.98153  -0.40061 

0.40615 

-0.36421 

17 

0.98153  -0.40061 

0.40615 

-0.36421 

22 

0.98153  -0.40061 

0.40615 

-0.36421 

16 

0.98153  -0.40061 

0.40615 

-0.36421 

11 

0.98153  -0.40061 

0.40615 

-0.36421 

26 

0.98153  -0.40061 

0.40615 

-0.36421 

6 

0.92613  -0.40061 

0.09581 

-0.36421 

4 

0.93956 

0.40061 

0.11880 

0.36421 

19 

0.93956 

0.40061 

0.11880 

0.36421 

13 

0.93956 

0.40061 

0.11880 

0.36421 

1 

0.93956 

0.40061 

0.11880 

0.36421 

8 

0.93956 

0.40061 

0.11880 

0.36421 

24 

0.93956 

0.40061 

0.11880 

0.36421 

9 

0.93956 

0.40061 

0.11880 

0.36421 

23 

0.93956 

0.40061 

0.11880 

0.36421 

20 

0.93956 

0.40061 

0.11880 

0.36421 

12 

0.93956 

0.40061 

0.11880 

0.36421 

3 

0.17260  -0.84562 

0.00710 

-0.81702 

18 

0.17260 

1.91407 

0.03639 

3.31157 

7 

0.17260  -0.84562 

0.00710 

-0.81702 

25 

0.17260 

0.25826 

0.00066 

0 . 23255 

14 

0.17260  -0.84562 

0.00710 

-0.81702 

10 

Diagnostics  Case  Statistics 
Actual  Predicted 
Value  Value  Residual  Leverage 
2.42  2.42108  -0.00108 

2.35  2.35108  -0.00108 

2.15  2.15108  -0.00108 

2.16  2.16108  -0.00108 

1.99  1.99108  -0.00108 

2.38  2.38108  -0.00108 

2.34  2.34108  -0.00108 

2.4  2.40108  -0.00108 

2.01  2.01108  -0.00108 
2.47  2.47108  -0.00108 

2.69  2.69217  -0.00217 

2.41  2.40804  0.00196 

2.15  2.14804  0.00196 

2.39  2.38804  0.00196 

2.22  2.21804  0.00196 

2.49  2.48804  0.00196 

2.16  2.15804  0.00196 

2.17  2.16804  0.00196 

2.24  2.23804  0.00196 

2.67  2.66804  0.00196 

2.26  2.25804  0.00196 

2.25  2.26532  -0.01532 

2.3  2.26532  0.03468 

2.26532  -0.01532 
2.26532  0.00468 


Student  Cook’s  Outlier  Run 
Residual  Distance  t  Order 
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B.3  Example  ANOVA  for  Total  Kills  Response 

Responsetotal  K 

ANOVA  for  Response  Surface  Linear  Model 
Analysis  of  variance  table  [Pairtial  sum  of  squares] 


Sum  of 

Mean 

Source 

Squares 

DF 

Square 

Model 

0.13676 

5 

0.0274 

A 

0.06395 

1 

0.0640 

B 

0.00982 

1 

0.0098 

C 

0.02563 

1 

0.0256 

D 

0.02518 

1 

0.0252 

E 

0.00652 

1 

0.0065 

ResidualO . 14286 

20 

0.0071 

Lack  of 

0.13658 

16 

0.0085 

Pure  ErrO. 00628 

4 

0.0016 

Cor  TotaO. 27962 

25 

F 


Value 

Prob  >  F 

3.8293 

0.0135 

signif  icauit 

8.9531 

0.0072 

1.3753 

0.2547 

3.5878 

0.0728 

3.5249 

0.0751 

0.9122 

0.3509 

5.4370 

0.0566 

not  significant 

The  Model  F-*value  of  3.83  implies  the  model  is  significant.  There  is  only 
a  1.35*/,  chance  that  a  "Model  F-Value"  this  large  could  occur  due  to  noise. 

Values  of  "Prob  >  F"  less  than  0.0500  indicate  model  terms  are  significant. 

In  this  case  A  are  significant  model  terms. 

Values  greater  than  0.1000  indicate  the  model  terms  are  not  significant. 

If  there  are  many  insignificant  model  terms  (not  counting  those  required  to  support  hierarchy) , 
model  reduction  may  improve  your  model. 

The  "Lack  of  Fit  F-value"  of  5.44  implies  there  is  a  5.66*/,  chance  that  a  "Lack  of  Fit  F- 
value"  this  large  could  occur  due  to  noise.  Lack  of  fit  is  bad  —  we  want  the  model  to  fit. 


Std.  Dev  0.08452 
Mean  3.00385 
C.V.  2.81357 
PRESS  0.24963 


R-Square  0.48910 
Adj  R“Sq  0.36137 
Pred  R-S  0.10725 
Adeq  Pre  7.72702 


The  "Pred  R-Squared"  of  0.1073  is  not  as  close  to  the  "Adj  R-Squared"  of  0.3614  as  one  might 
normally  expect.  This  may  indicate  a  large  block  effect  or  a  possible  problem  with  your  model 
and/or  data.  Things  to  consider  are  model  reduction,  response  tranf ormation,  outliers,  etc. 

"Adeq  Precision"  measures  the  signal  to  noise  ratio.  A  ratio  greater  than  4  is  desirable.  Your 
ratio  of  7.727  indicates  an  adequate  signal.  This  model  can  be  used  to  navigate  the  design  spac 
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Coefficient 

Standard  957.  Cl 

957.  Cl 

Factor  Estimate 

DF 

Error  Low 

High 

VIF 

Intercep  3.0064 

1 

0.0167  2.9715 

3.0412 

A-RDOT  -0.0608 

1 

0.0203  -0.1032 

-0.0184 

1.0172 

B-TOF  -0.0238 

1 

0.0203  -0.0662 

0.0186 

1.0172 

C-Prior  -0.0385 

1 

0.0203  -0.0809 

0.0039 

1.0172 

D-Engage  0.0382 

1 

0.0203  -0.0042 

0.0805 

1.0172 

E-Comm  R  0.0194 

1 

0.0203  -0.0230 

0.0618 

1.0172 

Final  Equation  in  Terms 


of  Coded  Factors: 


total  K  = 
3.006368 
-0.0608  *  A 
-0.02383  *  B 
-0.03849  *  C 
0.038153  ♦  D 
0.019408  ★  E 


Final  Equation  in  Terms 


of  Actual  Factors: 


total  K  = 
3.222475 
-0.30402  ♦ 
-0.11916  * 
-0.19246  * 
0.190764  * 
3.88E-06  ♦ 


ROOT 
TOF 
Prior 
Engage 
Comm  Range 


Diagnostics  Case  Statistics 

StauidardActual  Predicted  Student  Cook’s  Outlier  Run 

Order  Value  Value  ResidualLeverageResidualDistancet  Order 


1 

2.97 

2.9790 

-0.0090 

0.3058 

-0.1274 

0.0012 

-0.1242 

5 

2 

2.94 

2.9496 

-0.0096 

0.3058 

-0.1370 

0.0014 

-0.1336 

21 

3 

3.07 

2.9861 

0.0839 

0.3058 

1.1914 

0.1042 

1.2047 

2 

4 

2.81 

2.8257 

-0.0157 

0.3058 

-0.2227 

0.0036 

-0.2173 

15 

5 

2.87 

2.9415 

-0.0715 

0.3058 

“1.0151 

0.0756 

-1.0159 

17 

6 

3.1 

3.0654 

0.0346 

0.3058 

0.4907 

0.0177 

0.4812 

22 

7 

3.16 

3.1101 

0.0499 

0.3058 

0.7090 

0.0369 

0.6999 

16 

8 

3.21 

3.1394 

0.0706 

0.3058 

1.0027 

0.0738 

1.0028 

11 
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9 

2.95 

2.9122  0.0378 

0.3058  0.5374 

0.0212  0.5276 

26 

10 

3.1 

3.0236  0.0764 

0.3058  1.0850 

0.0864  1.0901 

6 

11 

3.07 

3.0719  “0.0019 

0.4396  -0.0306 

0.0001  -0.0298 

4 

12 

2.95 

3.1171  -0.1671 

0.2412  -2.2698 

0.2729  -2.5676 

19 

13 

2.9 

2.8956  0.0044 

0.2203  0.0585 

0.0002  0.0570 

13 

14 

3.03 

3.0498  -0.0198 

0.2412  -0.2685 

0.0038  -0.2622 

1 

15 

2.93 

2.9630  -0.0330 

0.2203  -0.4418 

0.0092  -0.4327 

8 

16 

3.09 

3.0765  0.0135 

0.2412  0.1838 

0.0018  0.1793 

24 

17 

2.84 

2.9363  -0.0963 

0.2203  -1.2900 

0.0784  -1.3132 

9 

18 

2.89 

2.9369  -0.0469 

0.2412  -0.6368 

0.0215  -0.6271 

23 

19 

2.93 

3.0759  -0.1459 

0.2203  -1.9544 

0.1799  -2.1179 

20 

20 

3.16 

2.9710  0.1890 

0.2412  2.5669 

0.3490  3.0553 

12 

21 

3.14 

3.0417  0.0983 

0.2203  1.3171 

0.0817  1.3433 

3 

22 

2.99 

3.0064  -0.0164 

0.0390  -0.1976 

0.0003  -0.1927 

18 

23 

3.06 

3.0064  0.0536 

0.0390  0.6473 

0.0028  0.6377 

7 

24 

2.96 

3.0064  -0.0464 

0.0390  -0.5597 

0.0021  -0.5498 

25 

25 

3.01 

3.0064  0.0036 

0.0390  0.0438 

0 . 0000  0 . 0427 

14 

26 

2.97 

3.0064  -0.0364 

0.0390  -0.4390 

0.0013  -0.4299 

10 
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B.4  Example  ANOVA  for  Total  Hits  Response 


Response:  total  H 

ANOVA  for  Response  Surface  Quadratic  Model 
Analysis  of  variance  table  [Partial  sum  of  squares] 


Sum  of 

Mean 

F 

Source 

Squares 

DF 

Square 

Value 

Prob  >  F 

Model 

0.54357 

20 

0.02718 

57.1962 

0.0001 

significant 

A 

0.11281 

1 

0.11281 

237.4095<  0.0001 

B 

0.01531 

1 

0.01531 

32.2246 

0.0024 

C 

0.05951 

1 

0.05951 

125.2418<  0.0001 

D 

0.00781 

1 

0.00781 

16.4411 

0.0098 

E 

0.01051 

1 

0.01051 

22.1231 

0.0053 

A2 

0.00006 

1 

0.00006 

0.1335 

0.7298 

B2 

0.00220 

1 

0.00220 

4.6369 

0.0839 

C2 

0.00082 

1 

0.00082 

1.7159 

0.2472 

D2 

0.01230 

1 

0.01230 

25.8880 

0.0038 

E2 

0.02247 

1 

0.02247 

47.2791 

0.0010 

AB 

0.01878 

1 

0.01878 

39.5243 

0.0015 

AC 

0.00006 

1 

0.00006 

0.1321 

0.7312 

AD 

0.00266 

1 

0.00266 

5.5998 

0.0642 

AE 

0.00061 

1 

0.00061 

1.2839 

0.3086 

BC 

0.00013 

1 

0.00013 

0.2788 

0.6201 

BD 

0.00287 

1 

0.00287 

6.0311 

0.0575 

BE 

0.00110 

1 

0.00110 

2.3203 

0.1882 

CD 

0.00016 

1 

0.00016 

0 . 3429 

0.5836 

CE 

0.00695 

1 

0.00695 

14.6215 

0.0123 

DE 

0.00080 

1 

0.00080 

1.6757 

0.2521 

Residual 

0.00238 

5 

0.00048 

Lack  of  Fit 0.00 146 

1 

0.00146 

6.33002 

0.0656 

not  significant 

Pure  Error 

0.00092 

4 

0.00023 

Cor  Total 

0.54595 

25 

The  Model  F-value  of  57.20  implies  the  model  is  significant.  There  is  only 
a  O.Oiy,  chance  that  a  "Model  F-Value"  this  large  could  occur  due  to  noise. 

Values  of  "Prob  >  F"  less  than  0.0500  indicate  model  terms  are  significant. 

In  this  case  A,  B,  C,  D,  E,  D'^2,  £*"2,  AB,  CE  are  significant  model  terms. 

Values  greater  than  0.1000  indicate  the  model  terms  are  not  significant. 

If  there  are  many  insignificant  model  terms  (not  counting  those  required  to  support  hierarchy) , 
model  reduction  may  improve  your  model. 
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The  "Lack  of  Fit  F-value*'  of  6.33  implies  there  is  a  6.56*/,  chance  that  a  "Lack  of  Fit  F- 
value"  this  large  could  occur  due  to  noise.  Lack  of  fit  is  bad  we  want  the  model  to  fit. 


Std.  Dev.  0.02180 
Mean  5.87827 
C.V.  0.37083 
PRESS  1.05054 


R-Square  0.99565 
Adj  R-Sq  0.97824 
Pred  R-S-0. 92426 
Adeq  Pre26. 26568 


A  negative  "Pred  R-Squared"  implies  that  the  overall  mean  is  a  better  predictor  of  your 
response  than  the  current  model. 


"Adeq  Precision"  measures  the  signal  to  noise  ratio.  A  ratio  greater  than  4  is  desirable.  Your 
ratio  of  26.266  indicates  an  adequate  signal.  This  model  can  be  used  to  navigate  the  design  spa 


Coefficient 

Stemdard  95*/.  Cl  95*/.  Cl 

Factor 

Estimate 

DF 

Error  Low  High 

VIF 

Intercept  5.87768 

1 

0.00906  5.85440  5.90096 

A-RDOT 

-0.13041 

1 

0.00846  -0.15217-0.108652.65251 

B-TOF 

-0.04805 

1 

0.00846  -0.06980-0.026292.65251 

C“Prior 

-0.09472 

1 

0.00846  -0.11648-0.072962.65251 

D-Engage 

0.03432 

1 

0.00846  0.01256  0.05608 

2.65251 

1-Comm  Rang-0. 03981 

1 

0.00846  -0.06157-0.018052.65251 

A2 

-0.00175 

1 

0.00479  -0.014060.01056 

1.01524 

B2 

0.01031 

1 

0.00479  -0.002000.02262 

1.01524 

C2 

-0.00627 

1 

0.00479  -0.018580.00604 

1.01524 

D2 

-0.02436 

1 

0.00479  -0.03667-0.012051.01524 

E2 

0.03292 

1 

0.00479  0.02062  0.04523 

1.01524 

AB 

-0.07839 

1 

0.01247  -0.11045-0.046343.58667 

AC 

0.00453 

1 

0.01247  -0.027520.03658 

3.58667 

AD 

-0.02951 

1 

0.01247  -0.061560.00255 

3.58667 

AE 

-0.01413 

1 

0.01247  -0.046180.01792 

3.58667 

BC 

-0.00658 

1 

0.01247  -0.038640.02547 

3.58667 

BD 

-0.03062 

1 

0.01247  -0.062680.00143 

3.58667 

BE 

-0.01899 

1 

0.01247  -0.051050.01306 

3.58667 

CD 

0.00730 

1 

0.01247  -0.024750.03935 

3.58667 

CE 

0.04768 

1 

0.01247  0.01563  0.07973 

3.58667 

DE 

0.01614 

1 

0.01247  -0.015910.04819 

3.58667 

Final  Equation  in  Terms  of  Coded  Factors: 


total  H  = 
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5.877683 
‘“0.13041  *  A 
-0.04805  *  B 
-0.09472  *  C 
0.034319  *  D 
-0.03981  *  E 
-0.00175  *  A2 
0.010311  ♦  B2 
-0.00627  *  C2 
-0.02436  ♦  D2 
0.032924  ♦  E2 
-0.07839  *  A  *  B 
0.004532  ♦  A  *  C 
-0.02951  ♦  A  *  D 
-0.01413  ♦  A  *  E 
-0.00658  ★  B  *  C 
-0.03062  *  B  *  D 
-0.01899  *  B  ♦  E 
0.007301  ♦  C  ♦  D 
0.04768  ♦  C  *  E 
0.016141  *  D  *  E 

Final  Equation  in  Terms  of  Actual  Factors: 

total  H  = 

5.591656 
1.092231  *  ROOT 
1.374346  ♦  TOF 
-0.84097  *  Prior 
1.533489  ♦  Engage 
-5.3E-05  *  Comm  Range 
-0.04374  ♦  RD0T2 
0.257771  ♦  T0F2 
-0.15681  ♦  Prior2 
-0.60907  *  Engage2 
1.32E-09  *  Comm  Range2 
-1.9598  *  ROOT  *  TOF 
0.113288  ♦  RDOT  *  Prior 
-0.73768  *  RDOT  *  Engage 
-1.4E-05  *  RDOT  *  Comm  Range 
-0.16459  *  TOF  *  Prior 
-0.76555  *  TOF  *  Engage 


B-11 


-1.9E*'05  ♦  TOP  *  Comm  Range 
0.182534  ♦  Prior  *  Engage 
4.77E-05  *  Prior  *  Comm  Range 
1.61E-05  *  Engage  *  Comm  Range 


Diagnostics  Case  Statistics 


Standard 

Actual 

Predicted 

Student  Cook’s 

Outlier 

Run 

Order 

Value 

Value 

Re sidualLeverageResidualDi stance  t 

Order 

1 

5.805 

5.79981 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

5 

2 

5.82 

5.81481 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

21 

3 

5.895 

5.88981 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

2 

4 

5.6 

5.59481 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

15 

5 

5.63 

5.62481 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

17 

6 

5.935 

5.92981 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

22 

7 

6.06 

6.05481 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

16 

8 

6.11 

6.10481 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

11 

9 

5.76 

5.75481 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

26 

10 

5.97 

5.96481 

0.00519  0.98153 

1.75040  7.75407 

2.51595 

6 

11 

6.075 

6.06463 

0.01037  0.92613 

1.75040  1.82909 

2.51595 

4 

12 

6.1 

6.10938 

-0.009380.93956 

-1.750402.26798 

-2.51595 

19 

13 

5.625 

5.63438 

-0.009380.93956 

-1.750402.26798 

-2.51595 

13 

14 

5.99 

5.99938 

-0.009380.93956 

-1.750402.26798 

-2.51595 

1 

15 

5.815 

5.82438 

-0.009380.93956 

-1.750402.26798 

-2.51595 

8 

16 

6.02 

6.02938 

-0.009380.93956 

-1.750402.26798 

-2.51595 

24 

17 

5.675 

5.68438 

-0.009380.93956 

-1.750402.26798 

-2.51595 

9 

18 

5.725 

5.73438 

-0.009380.93956 

-1.750402.26798 

-2.51595 

23 

19 

5.85 

5.85938 

-0.009380.93956 

-1.750402.26798 

-2.51595 

20 

20 

6.05 

6.05938 

-0.009380.93956 

-1.750402.26798 

-2.51595 

12 

21 

5.905 

5.91438 

-0.009380.93956 

-1.750402.26798 

-2.51595 

3 

22 

5.89 

5.87768 

0.01232  0.17260 

0.62115  0.00383 

0.57834 

18 

23 

5.89 

5.87768 

0.01232  0.17260 

0.62115  0.00383 

0.57834 

7 

24 

5.88 

5.87768 

0.00232  0.17260 

0.11683  0.00014 

0.10464 

25 

25 

5.9 

5.87768 

0.02232  0.17260 

1.12548  0.01258 

1 . 16499 

14 

26 

5.86 

5.87768 

-0.017680.17260 

-0.891820.00790 

-0.86985 

10 

B-12 


B.5  Example  ANOVA  for  Target  Formula  Response 


Re  sp  on  s  ef  omml  a 

ANOVA  for  Response  Surface  Linear  Model 
Analysis  of  variance  table  [Partial  sum  of  squares] 


Sum  of 

Mean 

F 

Source  Squares 

DF 

Square 

Value 

Prob  >  F 

Model 

1.2867 

5 

0.2573 

9.3546 

0.0001  significant 

A 

0.9424 

1 

0.9424 

34.2578 

<  0.0001 

B 

0.2934 

1 

0.2934 

10.6670 

0.0039 

C 

0.1631 

1 

0.1631 

5.9280 

0.0244 

D 

0.0018 

1 

0.0018 

0.0658 

0.8002 

E 

0.0063 

1 

0.0063 

0.2308 

0.6362 

Residual 

0.5502 

20 

0.0275 

Lack  of 

0.5215 

16 

0.0326 

4.5375 

0.0769  not  significant 

Pure  Err 

0.0287 

4 

0.0072 

Cor  Tota 

1.8369 

25 

The  Model  F-value  of  9.35  implies  the  model  is  significant.  There  is  only 
a  O.Or/,  chance  that  a  "Model  F~Value"  this  large  could  occur  due  to  noise. 

Values  of  "Prob  >  F"  less  than  0.0500  indicate  model  terms  are  significant. 

In  this  case  A,  B,  C  are  significant  model  terms. 

Values  greater  than  0.1000  indicate  the  model  terms  are  not  signif iceint . 

If  there  are  many  insignificant  model  terms  (not  counting  those  required  to  support  hierarchy) , 
model  reduction  may  improve  your  model. 

The  "Lack  of  Fit  F-value"  of  4,54  implies  there  is  a  7.69*/,  chance  that  a  "Lack  of  Fit  F- 
value"  this  large  could  occur  due  to  noise.  Lack  of  fit  is  bad  —  we  want  the  model  to  fit. 


Std.  Dev  0.16586 
Mean  8.02558 
C.V.  2.06662 
PRESS  0.99770 

The  "Pred  R-Squared" 


R-Square  0.70048 
Adj  R-Sq  0.62560 
Pred  R-S  0.45684 
Adeq  Prell. 56597 

0.4568  is  in  reasonable 


agreement  with  the  "Adj 


R-Squared"  of  0.6256. 


"Adeq  Precision"  measures  the  signal  to  noise  ratio.  A  ratio  greater  than  4  is  desirable.  Your 
ratio  of  11.566  indicates  an  adequate  signal.  This  model  can  be  used  to  navigate  the  design  spa 


Coefficient 
Factor  Estimate  DF 


Standard  95*/,  Cl  95*/,  Cl 
Error  Low  High 


VIF 
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Intercep  8.0436 

1 

0.0328  7.9753 

8.1120 

A-RDOT  -0,2334 

1 

0.0399  -0.3166 

-0.1502 

1.0172 

B-TOF  -0.1302 

1 

0.0399  -0.2134 

-0.0471 

1.0172 

C-Prior  -0.0971 

1 

0.0399  -0.1803 

-0.0139 

1.017^ 

D-Engage  0.0102 

1 

0.0399  -0.0730 

0.0934 

1.0172 

E-Comm  R-0.0192 

1 

0.0399  -0.1023 

0.0640 

1.0172 

Final  Equation  in  Terms 


of  Coded  Factors: 


formula  = 
8.043642 
-0.23342  ★  A 
-0.13025  ♦  B 
-0.0971  ♦  C 
0.010229  *  D 
-0.01916  *  E 


Final  Equation  in  Terms 


of  Actual  Factors: 


formula 


9.433561 
-1.16709  * 
-0.65124  * 
-0.48549  * 
0.051147  ♦ 
-3.8E-06  ♦ 


ROOT 
TOF 
Prior 
Engage 
Comm  Range 


Diagnostics  Case  Statistics 


Standard  Actual 

Predicted 

Student 

Cook's 

Outlier 

Run 

Order 

Value 

Value 

ResidualLeverageResidualDistance  t 

Order 

1 

7.78 

7.8065 

-0.0265 

0.3058 

-0.1915 

0.0027 

-0.1868 

5 

2 

7.96 

7.8728 

0.0872 

0.3058 

0.6313 

0.0293 

0.6215 

21 

3 

8.005 

8.0203 

-0.0153 

0.3058 

-0.1109 

0.0009 

-0.1081 

2 

4 

7.68 

7.5918 

0.0882 

0.3058 

0.6382 

0.0299 

0.6285 

15 

5 

7.715 

7.7477 

-0.0327 

0.3058 

-0.2365 

0.0041 

“0.2309 

17 

6 

7,955 

8.0286 

-0.0736 

0.3058 

-0.5329 

0.0208 

-0.5231 

22 

7 

8.425 

8.3013 

0.1237 

0.3058 

0.8952 

0.0588 

0.8906 

16 

8 

8.54 

8.2350 

0.3050 

0.3058 

2.2072 

0.3576 

2.4735 

11 

9 

00 

7.8140 

0.2860 

0.3058 

2.0696 

0.3144 

2.2756 

26 

10 

7.985 

8.0791 

-0.0941 

0.3058 

-0.6809 

0.0340 

-0.6715 
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11 

8.35 

8.5133  -0.1633 

0.4396  -1.3155 

0.2263  -1.3416 

4 

12 

8.67 

8.4687  0.2013 

0.2412  1.3930 

0.1028  1.4289 

19 

13 

7.725 

7.6186  0.1064 

0.2203  0.7269 

0.0249  0.7180 

13 

14 

8.25 

8.2808  -0.0308 

0.2412  -0.2135 

0.0024  -0.2083 

1 

15 

7.795 

7.8064  -0.0114 

0.2203  -0.0781 

0.0003  -0.0761 

8 

16 

8.41 

8.2205  0.1895 

0.2412  1.3118 

0.0912  1.3374 

24 

17 

7.8 

7.8668  -0.0668 

0.2203  -0.4562 

0.0098  -0.4470 

9 

18 

7.94 

8.0250  -0.0850 

0.2412  -0.5884 

0.0183  -0.5785 

23 

19 

7.89 

8.0623  -0.1723 

0.2203  -1.1763 

0.0652  -1.1884 

20 

20 

8.255 

8.0785  0.1765 

0.2412  1.2214 

0.0790  1.2375 

12 

21 

7.8 

8.0088  -0.2088 

0.2203  -1.4254 

0.0957  -1.4658 

3 

22 

7.955 

8.0436  -0.0886 

0.0390  -0.5452 

0.0020  -0.5354 

18 

23 

7.79 

8.0436  -0.2536 

0.0390  -1.5600 

0.0165  -1.6224 

7 

24 

7.985 

8.0436  -0.0586 

0.0390  -0.3607 

0.0009  -0.3527 

25 

25 

8 

8.0436  -0.0436 

0.0390  -0.2684 

0.0005  -0.2621 

14 

26 

7.905 

8.0436  -0.1386 

0.0390  -0.8527 

0.0049  -0.8466 
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